THE SECRET TO TALKING TO AN AI CHATBOT

WANT TO get the best answers out of an AI? It’s all in how you talk to it. Tell it, for instance, to pretend it is Albert Einstein. Or that somebody’s life depends on the response. Or that it needs to stay focused on its goals. Prodding an artificial-intelligence chatbot is nothing like doing a Google search. Instead, it is like having a conversation with a booksmart person who needs coaxing—sometimes very indirect or bizarre coaxing—to give the most creative and effective answers to questions. The trouble is, nobody knows why AI responds to those strange prompts in the ways it does—not even the people who created large language models such as OpenAI’s ChatGPT, Google’s Gemini and Microsoft’s Copilot. That has led to experiments and guesswork to find the approaches that work best. Some researchers even ask the chatbots themselves for tips on how to talk to them. Below are some creative ways researchers have found to elicit better chatbot responses. These aren’t universal truths; not all of them work with all chatbots or with all types of questions. But Asking a chatbot to act as an ex[1]pert—even a long-dead one—can yield better results, or at least be more entertaining. If you’re seeking information about designing a mid-century modern house, tell it to “act as architect Frank Lloyd Wright.” Here’s how Gemini began its response, which listed five useful tips: “Alright, here’s Frank Lloyd Wright speaking to you about your future home…” Similarly, if you’re trying to understand Einstein’s general theory of relativity, ask it to answer as Einstein would. ChatGPT’s response to that question: “Ah, mein freund, I shall endeavour to elucidate the wondrous fabric of space-time as perceived through the lens of my general theory of relativity…” It then explained the theory in fairly easy-to-understand terms. Why does this technique work? “If you ask the AI to ‘imagine you’re a world-renowned chef,’ you’re signalling that you want a response that reflects culinary expertise,” Copilot responded when I asked it the question. “The AI will then generate a response that aligns with what it has learned about how chefs talk and the kind of information they might provide.” At the other end of the scale, telling a chatbot to act like a middle-school teacher should produce simple explanations for hard-to[1]understand concepts, such as how Continued from pageR1 nuclear power is created or the techniques used in gene editing. Encourage the chatbot to do better Telling a chatbot that its response is important to your job, or to take pride in its work, can improve its performance, a group of re[1]searchers found. The researchers, from Micro[1]soft and several universities, created 11 motivating phrases de[1]rived from human psychology, including, “Stay focused and dedicated to your goals” and “Are you sure this is your final answer?” They inserted these phrases into questions or tasks they gave to six chatbots. The outcome: The method, which the group dubbed “Emotion Prompt,” boosted the accuracy and usefulness of the responses. The chatbots showed a 115% improvement on an artificial-intelligence test called BIG-Bench—a compendium of over 200 extremely difficult language-based tasks—over straightforward prompts. Chatbots “possess emotional intelligence and can be enhanced by emotional stimuli,” the group said. Ask the AI to suggest prompts Using specialized software called an automatic prompt optimizer, researchers at the tech company Broadcom asked a chat[1]bot for phrases that would boost the accuracy of the answers it gives on a math test. Among other things, the chat[1]bot suggested that researchers use “Star Trek” language in their questions: “Command, we need you to plot a course through this turbulence and locate the source of the anomaly. Use all available data and your expertise to guide us through this challenging situation.” And the chatbot said it would answer in this format: “Captain’s Log, Stardate [insert date here]: We have successfully plotted a course through the turbulence and

 are now approaching the source of the anomaly.” When the researchers used this verbiage to run the test, the chat[1]bot followed it with its numerical answer to the question. This odd[1]ball role-playing improved the score to 59% correct answers, from 56% correct answers when researchers didn’t use prompts the AI helped create. “Surprisingly, it appears that the [AI] model’s proficiency in mathematical reasoning can be enhanced by the expression of an affinity for ‘Star Trek,’ ” the re[1]searchers wrote. They called the Chabot’s suggested prompt language “remarkably different from anything a human practitioner would be likely to generate.” “The takeaway here is not to try ‘Star Trek’-style prompting for other problems,” says one of the researchers, Rick Battle, because the chatbot suggested that word[1]ing specifically for this test. “The takeaway is that no human-writ[1]ten prompts will outperform an automatic prompt optimizer.”

Battle couldn’t explain why the “Star Trek” role-playing would make the chatbot more accurate. “This is just another example of LLMs being a black box,” he says. Play a role yourself Instead of asking the chatbot to play a role, try playing one yourself. If you’re asking a medical question, “you could role-play as a concerned parent, a sceptical patient, or even a curious alien studying human biology,” ChatGPT told me when I asked it how to obtain better responses. “This approach can sometimes elicit more detailed and empathetic responses,” it said. Be genial, and don’t get mad Chatbots seem to work better if you ask them things in a colloquial and friendly manner, but don’t go overboard. “Excessive flattery is not necessarily welcome,” according to researchers at Waseda University in Tokyo, who tested the impact of politeness levels in prompts. They found chatbot performance is “strongly related to human behaviour” and that they “are sensitive and vulnerable to prompts.” The researchers devised questions that used varying degrees of politeness and respect, from the lowest, level 1, to the highest, level 8. They used these questions to prompt answers from various chatbots. With ChatGPT version 3.5, the researchers found that when using level 8 of polite language, the chatbot scored 60.02 on a language understanding test, compared with the score of 51.93 for questions using level 1 of politeness. “However, highly respectful prompts do not always lead to better results. In most conditions, moderate politeness is better,” the researchers wrote. The same study found that be[1]coming angry with a chatbot that isn’t responding as you would like can make things worse. “Using impolite prompts can result in the low performance of LLMs, which may lead to increased bias, incorrect answers or refusal of answers,” the researchers wrote. Encourage it to be methodical A study by Google found that telling a chatbot to “take a deep breath and work on this problem step by step” produced markedly better answers to math questions. Other effective phrases—created largely by chatbots themselves— included “break this down” and “a little bit of arithmetic and a logical approach will help us quickly arrive at the solution to this problem.” Chatbots primed with these encouraging words outperformed human-designed prompts by up to 8% on a test of grade-school math word problems, and by up to 50% on the BIG-Bench tasks, the re[1]searchers said.