What is context rot?

Context rot is the drop in an AI model’s accuracy as more text is fed into it at once. Even well within the model’s limit, important details get lost among less important ones, and answers become slightly wrong. It shows up in long chat sessions, in very long documents and in AI agents that collect a lot of material over many steps.

Why it matters for a business

A model that accepts a million tokens does not use a million tokens equally well. Pasting everything into a prompt feels thorough and often produces a worse answer than a short, relevant prompt would, more slowly and at a higher cost. The errors are the hard kind to catch: close to right rather than obviously wrong. The remedy is choosing what goes into the window rather than how much fits, which is most of the work in any AI product that runs for a long time.

Example

Let’s say someone pastes a 300-page supplier agreement into an assistant and asks for the late-delivery penalty. The answer quotes a penalty clause from the wrong schedule. Asked again with only the delivery section pasted in, the same model gets it right.

Go deeper

What a context window is, and what happens when it fills up

Related terms

• Context window
• Context engineering
• Token
• RAG (retrieval-augmented generation)

All terms, A to Z