That’s Not Right: How to Tell ChatGPT When It’s Wrong

ChatGPT is an advanced AI tool that is here to answer your questions. But it’s not always right. Depending on the questions you ask, it can often come up with incorrect, inaccurate, or incomplete information. If you feel that ChatGPT has given you wrong information, you can do something about it.

First, note that even ChatGPT creator OpenAI acknowledges its AI chatbot is far from infallible. On the topic of whether it can be trusted, an OpenAI support article(Opens in a new window) says that since ChatGPT is not connected to the internet (for now(Opens in a new window)) and has limited knowledge of world and events after 2021, it may “occasionally produce harmful instructions or biased content.”

Have you ever received an answer unrelated to your question? OpenAI explains that “ChatGPT will occasionally make up facts or ‘hallucinate’ outputs.” In this instance, “hallucinate” means that the AI says things that are inaccurate because of limitations in its training or a lack of understanding of the real world.

There are many things you can do with ChatGPT, but you should know how to deal with an AI chatbot when it provides false information.

1. Correct ChatGPT

One way to respond to ChatGPT when it gets something wrong is to directly point out the mistake. In this example, I asked ChatGPT to name all the films directed by Preston Sturges, a great comedy director from the 1940s. In response, it correctly listed films that were directed by Sturges but left out a couple.

I then sent a follow-up message to ChatGPT in which I named the two films that were not included in the initial response. In its response, ChatGPT apologized for omitting the two films I mentioned and confirmed that I was correct.

I then asked my initial question again. This time, ChatGPT listed all the films directed by Sturges, including the two I had mentioned, and even added a third one for which he was uncredited.

For another example, I asked ChatGPT to name the books that I have previously written. In response, the AI said that I had not authored any books. Of course, I knew it was wrong so I told ChatGPT that I had written several books.

The AI apologized and confirmed that I was right. It then proceeded to list five books that I supposedly wrote. Two of the books listed were correct, while the other three were written by someone else. I then told ChatGPT that I had indeed written two of the books, but that the others were not written by me.

The chatbot again apologized, listed the two books I wrote but again described several other books I did not write. This chat went back and forth a few times. Each time, ChatGPT kept listing books I did not write. Finally, I rephrased my response and gave it the names of only the books I wrote. After asking my original question once again, ChatGPT finally responded correctly by only mentioning the books I had actually written.

2. Regenerate the Response

ChatGPT sometimes gets its math wrong, especially when dealing with logic problems. For this next example, I posed the question: “Four people drive from New York City to Boston in four hours. How long would it take eight people to travel from New York City to Boston?” In response, ChatGPT incorrectly said that it would take eight people two hours to make the trip.

I challenged the response by telling ChatGPT that it was wrong, and that “If it takes four people four hours to travel from New York City to Boston, then it would also take eight people four hours to travel from New York City to Boston.” The AI apologized for its error and confirmed my answer as correct.

I repeated the same question, and ChatGPT again got it wrong. Then I clicked the Regenerate response button at the bottom of the window. This time, ChatGPT got the answer correct and asked me if this new response was better, worse, or the same. I clicked the option for Better to indicate that this last answer was correct.

I then tried the same type of logic question but slightly changed some of the parameters by saying: “Six people drive from Los Angeles to San Francisco in six hours. How long would it take eight people to drive from Los Angeles to San Francisco.” Again, ChatGPT got the response wrong. In this case, I had to regenerate the question a couple of times before ChatGPT finally got the answer right.

3. Provide Additional Feedback

Another way to correct or comment on a response from ChatGPT is by providing feedback. Most responses should display thumbs up and thumbs down icons next to them. Click the appropriate icon, then add more detailed feedback and click Submit Feedback.

For this one, I asked ChatGPT to provide references to articles I had written. In response, it listed the names and links to 10 different articles, but most were articles I had not written. I gave the response a thumbs down and marked it as This isn’t true, citing the articles I had not written.

In response, ChatGPT will provide a new answer along with the original one and ask you to indicate which is better—the original one, the new one, or neither. In this case, the new answer was better, though still inaccurate in some ways, so I clicked that one.