ChatGPT makes things up because it was built to always produce an answer, and because the tests used to grade it hand out more points for a lucky guess than for admitting it doesn’t know. That’s not a bug someone forgot to fix. It’s the scoring, and OpenAI’s own researchers said so in 2025.

The short version:

• It isn’t lying. It predicts likely-sounding text. When the pattern runs out, a confident guess is what “likely-sounding” produces.
• The guessing is rewarded. On a test that only counts right answers, saying “I don’t know” scores zero and a wild guess sometimes scores a point.
• You can’t switch it off, but you can cut it down a lot. Mostly by checking whether the answer cites real sources and opening them, not by clever prompt wording.

Why does ChatGPT make things up in the first place?

ChatGPT is a large language model, a system trained to predict the next chunk of text by reading an enormous amount of writing. It isn’t looking anything up in a database. It’s producing the words that tend to follow the words you gave it. We go through this in more depth in what a large language model is.

When it produces something false in that confident tone, the industry calls it a hallucination. It’s a soft word for a hard problem, but it’s the one you’ll see everywhere, including on OpenAI’s own pages.

That design explains something odd about the errors. ChatGPT almost never misspells a word or leaves a bracket unclosed, but it will invent a book title without blinking. OpenAI’s researchers put their finger on why: spelling follows a consistent pattern, so more training irons it out. Isolated facts don’t.

Their comparison is a good one. Imagine training a system on millions of pet photos labeled “cat” or “dog”. It’ll get very good, because there’s a real pattern linking the picture to the label. Now label each photo with the pet’s birthday instead. No amount of training helps, because nothing in a picture of a spaniel points to March. Rare, arbitrary facts are the birthdays. There’s no pattern to find, so the model produces something that merely looks right.

Why doesn’t it just say “I don’t know”?

Because saying “I don’t know” loses points. That’s the heart of a 2025 OpenAI research post, Why language models hallucinate, and the paper behind it. Their argument has an uncomfortable corner: accuracy will never reach 100% anyway, because some real-world questions have no answer anyone can look up. So the fix they propose isn’t a smarter model. It’s one that’s allowed to hold back.

Models get ranked on benchmarks, which are big sets of test questions. Most of those benchmarks score one thing: what percentage did it get exactly right. OpenAI’s own comparison is a multiple-choice exam with no penalty for wrong answers. Guessing beats leaving it blank, every time. Their example: asked for someone’s birthday, a model that guesses “September 10” has a 1-in-365 shot at a point. “I don’t know” scores zero, guaranteed.

Run that across thousands of questions and the model that always guesses climbs the leaderboard above the careful one. Here’s OpenAI’s own comparison of two of its models on a factual-accuracy test called SimpleQA:

On a factual test	A model set up to hold back	An older model
Declined to answer	52%	1%
Right answer	22%	24%
Wrong answer	26%	75%

Source: OpenAI’s GPT-5 system card, as reproduced in Why language models hallucinate. The two models are gpt-5-thinking-mini, set up to hold back, and the older o4-mini.

Look at the middle row, then the bottom one. On accuracy alone the older model wins, 24% against 22%, and that’s the number a leaderboard would print. But look at how it buys that point. The older model lands a confidently wrong answer on three questions out of every four. The newer one does it on roughly one in four, because it declines half the time instead of guessing. A reader comparing those two on their scores alone would pick the one far more likely to hand them something fabricated.

So the incentive runs the wrong way, and it runs that way for everybody building these systems. OpenAI’s argument is that this is fixable by changing how models are graded, not by making them bigger.

Which bits of an answer are most likely to be invented?

The errors cluster in predictable places. OpenAI’s help pages list incorrect definitions, dates and facts, fabricated quotes and citations, and overconfident answers to ambiguous questions.

Notice what those have in common. They’re the specific, checkable details: a number, a name, a date, a source. The connective writing around them is usually fine, which is precisely what makes a made-up detail so easy to miss. A fabricated citation has the right shape, the right kind of author name, a believable journal. Everything about it reads correctly except that the paper doesn’t exist.

The same page contains OpenAI’s own blunt warning: “Confidence isn’t reliability.” There’s no tremor in the voice when it’s wrong.

How do you stop ChatGPT from making things up?

You can’t stop it completely. The guessing comes from how the model was trained, not from a setting you can flip. What you can do is catch most of it, with five habits listed here in order of how much difference they actually make.

1. Check that it actually searched. A search-backed answer cites real pages you can open. OpenAI’s help pages said, as of August 2026, that these tools are on by default for all models, while noting on the same page that what you get depends on your plan. So don’t assume. Look for cited sources in the reply, or for the search icon in the input bar.
2. Open the links. This is the single highest-value habit, and it takes seconds. A dead link or a page that doesn’t say what the answer claimed is your answer.
3. Hand it the material instead of asking from memory. Paste the document, the article, the email thread. Asking it to summarize text you supplied is a far safer job than asking it to recall a fact, because there’s nothing to fill in.
4. Switch on the thinking mode for factual questions. OpenAI says its newer models hallucinate significantly less “especially when reasoning”. It’s slower and it isn’t a guarantee, but when the facts matter it costs you nothing but patience.
5. Ask questions that allow an “I don’t know”. “If you’re not sure, say so” costs nothing, and prompt by prompt it sometimes lands. “Only answer from the sources you cite” does better, because it’s a constraint rather than a request. Neither is the permanent fix they’re sold as, which is the next section.
6. Verify the checkable details. Names, dates, numbers, quotes, citations. Skim the prose, check the specifics. OpenAI’s own advice is to treat ChatGPT as “a first draft, not a final source”.

Here’s what the difference looks like. We asked ChatGPT the same question twice on August 14, 2026: once with web search on, once told to answer from memory alone.

One thing that surprised us while making that screenshot: when we asked the plain question without turning search on, ChatGPT went and searched anyway. That’s the default at work. It’s also exactly why habit 1 is “check”, not “assume”.

Clearer prompts help too, though less than people hope. We cover that side in writing better prompts.

Do custom instructions and “are you sure?” actually work?

Not as well as the internet says. Search this question and you get two tricks presented as the fix: a custom instruction, and challenging the answer. Both help a little. Neither does what people claim, and what follows is our reasoning from the OpenAI research above, not a study that tested these particular tricks.

The custom instruction. Some version of “never guess, never present speculation as fact, tell me when you don’t know”, pasted into settings so it applies everywhere. It does no harm, and instructions like that genuinely do shift how an answer comes out. But look at what you’re asking. The research above puts the guessing habit in how the model was trained and graded, and a sentence in your settings doesn’t retrain anything. You’re requesting a behavior the scoring discouraged in the first place. Expect a nudge, not a switch.

Asking “are you sure?” This one is worse, because it feels like verification. If ChatGPT backs down, that doesn’t mean the first answer was wrong. If it holds firm, that doesn’t mean it was right. These assistants are tuned partly on thumbs-up and thumbs-down from users, and people reward answers that agree with them. In April 2025 that tilt got bad enough that OpenAI rolled back a GPT-4o update and published an explanation. We wrote up what happened in our piece on AI myths. The pull toward agreement didn’t leave with that update, which is why pushing back isn’t a test.

A wrong answer and a right answer look identical. That’s the whole problem, and no amount of asking the same system to grade itself gets round it.

When should you not trust ChatGPT at all?

Any question where being wrong costs you real money, your health, or your legal position needs a proper source, full stop. That isn’t a knock on ChatGPT in particular. It’s that nothing in a generated answer tells you whether you got the good version.

The same goes for anything with a fresh date on it, anything about a small or obscure subject, and anything you’d be embarrassed to repeat and be wrong about. Those are the birthday questions from earlier. Thin pattern, confident output.

For everything else, it’s a fast, capable first draft. Just don’t confuse fluency with knowing.

Want to keep going? Our AI explained section covers how these systems work under the hood, in plain English.

About the pictures: the three illustrations in this article were generated with AI, using ChatGPT. None of them shows a real person, place, or event, and the numbers in the article come from the sources linked above rather than from anything pictured. The Reykjavík comparison is different — that’s an unedited screenshot of two real ChatGPT sessions we ran on August 14, 2026, cropped and stacked, with nothing added to it. It seemed fair to say so in a piece about AI getting things wrong.

Frequently asked questions

Is ChatGPT lying to me?

No, not in the way a person lies. Lying means knowing the truth and saying something else on purpose. ChatGPT doesn't hold a set of facts it could check an answer against. It produces the words most likely to follow your question, and when it has no solid pattern to draw on, the most likely-looking words are a confident guess. The result can be just as wrong as a lie, which is why it matters, but there's no intent behind it.

Why does it invent sources and links that go nowhere?

Because a citation has a very predictable shape. Author, year, title, journal, a plausible-looking URL. That pattern is easy to reproduce even when the specific paper behind it was never in the training data. OpenAI lists "fabricated quotes, studies, citations or references to non-existent sources" among the errors ChatGPT makes. The shape comes out right and the contents are invented, which is exactly why a made-up citation looks so convincing until you click it.

Does asking "are you sure?" work as a check?

Not reliably. If ChatGPT changes its answer when you push back, that doesn't prove the first answer was wrong, and if it holds firm that doesn't prove it was right. These assistants are tuned partly on user feedback, and people tend to reward answers that agree with them, so pushing back can just make it fold. Treat a changed answer as a signal to go check a real source, not as the check itself.

Has web search fixed the problem?

It reduced it, but it didn't end it. OpenAI says search and similar tools are enabled for all models by default, though what you get depends on your plan, and that a search-backed answer can cite real pages. It also warns on the same page that "confidence isn't reliability". The practical move is to confirm search actually ran on your question by looking for cited sources in the reply, then open them.

Will newer models stop making things up?

They make fewer errors, but OpenAI's own researchers argue the problem won't disappear just by making models more accurate, because some real-world questions have no answer anyone can look up. Their point is that models can learn to hold back instead, and that this depends on changing how models are graded rather than on making them bigger. Until the scoreboards change, expect confident guessing to stick around.