What is a context window?

A context window is the maximum amount of text an AI model can take in and work with at one time, measured in tokens. Everything the model uses for a reply has to fit in it: the instructions, any documents, the conversation so far and the new question. The largest current models accept around a million tokens, roughly 750,000 words. Anything outside the context window is invisible to the model.

Why it matters for a business

Most moments when “the AI forgot what I said” are the context window at work. Products give the conversation only a slice of the window, so in a long session older messages get dropped or summarised, usually without saying so. A bigger window is not automatically better either: accuracy on the part that matters tends to fall as more goes in, and answers take longer. What goes into the window matters more than how much fits.

Go deeper

What a context window is, and what happens when it fills up

Related terms

• Token
• Context rot
• Context engineering
• AI memory
• RAG (retrieval-augmented generation)

All terms, A to Z