AI fundamentals
Why AI hallucinates (makes things up)
An AI hallucination is an answer a model makes up and states as fact: a figure, a name, a source or an event that is not real. It happens because a model has no way to tell a true sentence from a plausible one. Both come out of the same next-word prediction, in the same voice, at the same level of confidence. So hallucination is not a separate failure bolted onto an otherwise reliable machine. It is the machine working normally on a question where plausible and true come apart.
The mechanism
Section titled “The mechanism”A model produces the next word given everything before it. There is no separate fact-checker in the loop. The question it answers is “what word probably comes next?”, not “is the sentence I am producing true?”
Where the training data held thousands of paragraphs of one shape, the model completes that shape even when it was never shown the particular fact being asked for. It writes the most plausible filling, fluently, in the same voice as every accurate paragraph it has ever produced.
A true answer and a fabricated one come out of the same process. They look identical from outside because on the inside they are the same kind of thing.
Where the failure concentrates
Section titled “Where the failure concentrates”Hallucination is not spread evenly across questions.
Specific numbers and dates. The model is reliable on the shape of a sentence about a number. The digit itself is far less reliable, especially for figures that were not repeated widely across the training data.
Names and citations. “According to a recent paper by Singh and Patel” reads as authoritative, and the paper may not exist. The form of a citation is regular and easy to produce. Who wrote what, and where, and when, are individual facts the model often does not hold.
Anything recent. Every model has a knowledge cutoff, the date its training data ends. Past that date it is guessing from the patterns of what came before, and what models actually know goes deeper into that gap. The chat products now search the web for questions like this, which moves the failure rather than removing it. The answer is only as good as the page that came back, and a model handed nothing useful still writes the sentence.
Anything internal to a single company. A company’s own data was not in the public training set. The model has nothing specific to draw on, so it matches against how comparable companies have been described in public, and the specifics come out wrong.
Long, confident, detailed answers. In human writing, length and specificity are signs of effort and expertise. In a model’s output they are neither. A 2-paragraph answer with 7 specific details is no harder to produce than a one-line answer, and each extra detail is another chance to be wrong. Models do hedge, and the hedging is trained: it tracks topics the lab wanted handled carefully, medicine and law and money, not whether the name in this sentence was invented 30 words ago.
What reasoning models change
Section titled “What reasoning models change”Ask a current model something involved and it works through the problem before answering. That is a trained behaviour, covered in how AI learns, and it does catch some of this. A model that checks its own steps can notice that the figure it produced in step 2 does not match the one it used in step 4. On problems with several steps the current generation is meaningfully stronger than the one before it.
What the working cannot do is supply a source. Checking an argument against itself finds contradictions inside the argument. It cannot tell the model that the drug approval it just described never happened. Where the answer turns on a fact the model never held, more working buys a better-argued wrong answer.
Why grounding changes the picture
Section titled “Why grounding changes the picture”Put the actual approval letter in front of the model and ask the same question, and the mechanism shifts. The model is no longer reaching for patterns from training. It is summarising text sitting in its context window. The next-word prediction now runs against the document, and the most likely next words become the words that match it.
Putting the answer into the model’s view before asking the question is what every serious factual-AI product is built on. Pasting a document into ChatGPT, uploading a PDF, connecting a knowledge base, enterprise “AI search”: all variants of the same move. The industry words for it are retrieval and grounding, and tools and memory walks through how it is wired up in real systems.
Grounding does not make hallucination impossible. A model can still misread a source, mis-summarise it, or smooth over a contradiction inside it. The failure changes from inventing a fact out of nothing to misrepresenting something that exists, which is smaller and much easier to check. It also moves the weak point. When a grounded system answers wrongly, the usual cause is the wrong passage coming back, not the model, and a wrong passage produces an answer that looks exactly like a right one.
The shape of the risk
Section titled “The shape of the risk”Hallucination tracks two things: how much the model actually holds on the question, and how much checking happens before the output is used.
Some topics sit in the training data from a thousand angles. Common concepts, well-documented technical material, generic business writing. There the patterns are mostly correct and ordinary review catches the rest. Drafting an internal email about scheduling a meeting is that end.
At the other end the model holds nothing specific. The company’s own numbers, an event from last week, a person who is not a public figure, an obscure regulation. There it fabricates steadily unless the relevant text is put in front of it first. Quoting a customer back to themselves, citing a clause from a contract, stating last quarter’s revenue: that end.
The same model handles both. Newer models fail less often, and a reasoning model fails less often again where the middle of the answer can be checked. The shape does not change. What an answer is worth depends on what is being asked and what is in front of the model when it is asked, far more than on which model is answering.
The expensive mistakes are rarely the obvious ones. A fabricated answer that reads like a briefing note gets reviewed the same way a correct one does, and passes. Output right in form and wrong in a part nobody stops on is what “almost true” names, and it is where the cost lands.
Common questions
What is an AI hallucination?
An AI hallucination is an answer an AI tool makes up and states as fact: a figure, a name, a quote, a source or an event that is not real. It reads just as confidently as a correct answer, which is what makes it hard to spot. It happens because these tools write by predicting likely words, and nothing in that process checks whether the result is true.
What are examples of AI hallucinations?
Typical examples are a citation to a research paper that was never written, details of a drug approval that never happened, a company's sales figure invented in a believable range, or a wrong date or name in an otherwise sound answer. Most made-up details are specifics: numbers, names, citations, recent events and anything about one company's own business. The general explanation around them is usually right, which is why the invented detail slips through.
How often does AI hallucinate?
There is no single rate, because it depends heavily on the question. On common, well-documented topics a current model is usually right. On specific numbers, names, citations, recent events or a company's own information it makes things up far more often. Newer models do it less than older ones, and giving the model the source document with the question cuts it further.
How do you prevent AI hallucinations?
They cannot be removed entirely, but they can be made rarer and easier to catch. Give the model the source text, such as the policy, contract or report, along with the question, so it summarises that instead of guessing from its training. Have a person check any figure, name or citation before it is used. And avoid asking a model on its own for facts it could not have, such as last week's news or a company's internal numbers.