What an AI model actually knows

A model holds no facts the way a database does. It holds patterns absorbed during training, and those patterns produce true answers wherever they happen to line up with the truth. Some material it has read from a thousand angles, and it is usually right there. Some was never in its training data at all, and it answers there in the same voice, at the same length, with the same confidence.

What “knowing” means for a model

There are no lookup tables inside a model, no search index, no stored records. What it has are the patterns absorbed during training: the weights, settled by billions of small adjustments across trillions of examples. Those patterns let it finish sentences in ways that are often true.

When the model gets a fact right, it did not remember the fact. The patterns in its training data made the true answer also the most likely answer.

When it gets a fact wrong, the same thing happened, and something else was the most likely answer. Nothing inside separates “I am writing a true thing” from “I am writing a plausible-sounding thing”. The same loop produces both.

What the model has seen a lot of, and what it has never seen

How much material the model has read varies enormously from subject to subject, and that is what decides where it holds up.

Seen a great deal: common knowledge. Anything written about at length in public sources: basic science, history, mainstream code patterns, common business concepts, popular books, well-known companies. The model has read this material from many angles, so the patterns agree with each other. It is usually reliable here.

Seen less: specialised fields with good documentation. Medicine, law, finance, narrow technical fields. Solid material exists, and there is less of it. The model is usually right on the basics and shaky at the edges. It can misstate a legal precedent or a drug interaction with exactly the fluency it uses when it gets them right.

Seen very little: niche, recent or fast-changing subjects. New technologies, regional politics, small companies, people who are not public figures. Some material exists, not enough for the patterns to hold. This is where the model invents details most readily.

Never seen: the company itself. Its products, customers, internal processes, prices and last quarter’s numbers were never in the training data. There is nothing to retrieve. Asked anyway, the model writes a plausible answer built from patterns of other businesses that look like this one, and it is almost always wrong on the specifics.

An assistant that appears to know the business is not an exception to any of this. Products keep short notes from past conversations and put them into the context before the model sees the question, which is why a new chat opens already knowing the team size or the preferred format. The model has not changed; the product is passing it notes. Those notes hold a summary the software chose to keep. They do not hold last year’s Q3 sales.

The knowledge cutoff

Every model has a knowledge cutoff: the date its training data ends. After that date the model knows nothing. A model whose data closed before an election, a price change or a change of leadership has no idea any of it happened, unless someone brings the information to it with the question.

The model is not lagging slightly behind the news. Its training data stopped, and it stopped there with it. Anything current in an answer arrived from a search, a document or a paste, not from the model.

Models also do not state their own cutoff reliably. Ask one and the answer can be approximate, off by months, or vague. Treat the cutoff as a real constraint whatever the model says about it.

Why “I don’t know” is rare

A trained model has no strong way of recognising the edge of its own knowledge. It predicts the next word. Asked something outside its training, it does not stop and check whether it knows; it produces the most likely next words.

For some questions the most likely next words are an honest hedge: “I’m not sure”, “I don’t have specific information about that.” Instruction tuning and alignment both push models towards hedging when a question looks like one that calls for it. But the hedge is a trained pattern, not a flag raised by something inside that checked.

So a model can answer confidently where it has no business answering, and hedge about something it knows well. What decides it is whether the question looks like one to answer or one to hedge, not whether the model knows. An answer invented this way is a hallucination, and that chapter covers where the invention concentrates.

A reasoning model does not close this. The working it generates before answering runs through the same loop as the answer, so it can check its own arithmetic carefully and still be certain about a figure it invented earlier in that working.

Where the model is reliable and where it is not

A rough field guide, for the model on its own: no search, no documents, nothing attached.

Question type	Reliability	Why
Definitions, common concepts	High	Covered from many angles in training
Explanation of well-documented topics	High	Explained many times over in the data
Code patterns from popular languages	High	Huge amount of training data
Specific numbers (recent, niche, internal)	Low	Pattern completion fills in a plausible figure
Specific names, citations, dates	Low	Plausible ones are easy to generate
Recent events after the cutoff	None	Not in the training data
Anything specific to the company	None	Never in the training data
Subjective interpretations	Mixed	Sounds reasonable; there is no source to check it against

The general shape of a thing, how it works and why, is solid. Specifics are where the model invents: numbers, names, dates, anything true of one case only. The invented ones read exactly like the correct ones.

How real systems close the gap

Systems that deal in facts close the gap in 3 ways, usually in combination. Every serious product sold as AI-powered is some mix of them.

A person checks the output before it is used. Drafting tools, code suggestions, internal research summaries: the model produces, a person decides. This is the loosest of the 3. It costs human time on every output, and it fits almost any task.

The source text goes in with the question. The relevant text is put into the model’s context at the moment of the question, so the model summarises a source instead of guessing from training. Pasting a PDF into ChatGPT is the manual version. A system that pulls the right documents out of a knowledge base and feeds them in is the built version, and tools and memory walks through that machinery. This is what the industry calls grounding.

The model is only asked what its training supports. A tool that drafts generic email openers needs neither of the other 2: the material is common and a poor opener costs almost nothing. The same model asked to recite a company’s financial figures is the wrong tool for the job, at any level of effort.

These are not alternatives. Most real deployments run all 3 in different places. The question for any given workflow is which one matches what the model actually has.

A more accurate description

The description that produces the Q3 surprise is a smart, slightly fuzzy researcher: someone who might be off by a bit, but who looked something up. Nothing was looked up.

The accurate description is a writer who has read an enormous amount and works with no notes, no search, no calculator and no record of any particular company. It will produce coherent prose on almost any topic in almost any style. Truth, where it appears, is a side effect of the patterns lining up with reality, not something the model checked.

Held that way, the surprises move. Clean prose on a topic the model has read a thousand times is what the mechanism predicts, not a feat. A confidently invented name on a question with no source behind it is also what the mechanism predicts, not a glitch. The model gets more useful the moment it stops being asked to do the thing it was never built to do.