What the context window is
A model keeps no record of a session. On every turn it is handed one block of text and runs the next-word loop over whatever that block contains. The block has a maximum size, and that size is the context window.
So an instruction given earlier only shapes an answer if it is still inside the block on that turn. Nothing carries over on its own, and when something drops out the model does not notice it is gone.
How the window works on each turn
Section titled “How the window works on each turn”Every turn, the product assembles one block of text and sends the whole thing. It holds the latest message, the earlier messages in the chat, the model’s own earlier replies, any document uploaded or pasted, anything a search or a tool call pulled in, and a hidden set of instructions from the product (the “system prompt”) that tells it how to behave.
The model reads that block and writes the next reply. Then the block is thrown away and rebuilt from scratch on the following turn. Nothing persists inside the model in between. (How AI generates a response walks the whole path from keypress to answer.)
What a token is
Section titled “What a token is”A model does not read whole words. It reads short chunks, some of which are whole words and some of which are pieces. “The” is one chunk. “Fundamentals” is about 3. Those chunks are called tokens, and they are the unit everything about the window is quoted in.
The rough conversion for English is 4 tokens for every 3 words. A thousand-word memo is about 1,300 tokens. A 250-page book is around 130,000.
Tokens are also the unit of the bill. The companies that sell access to models charge by the token, on the way in and on the way out, and what that comes to for a real workflow is mostly a question of how much text each run carries.
See it fill up
Section titled “See it fill up”The window here is tiny so that the limit arrives in a few messages. A real one holds far more and behaves the same way. Text goes in until there is no room, and then something already in there has to go.
How big the window is now
Section titled “How big the window is now”The current flagship models from the major labs all take roughly a million tokens in one request, which is about 750,000 words, or a couple of thousand pages. Cheaper tiers and older releases hold a fraction of that, commonly a few hundred thousand.
What a product hands the model is a different number. Almost none of them spend the whole window on one conversation. Most allot a few thousand tokens to the chat history and keep the rest for the system prompt, uploaded files, search results and tool output. That is why a long session runs out of room long before the model would, and why a morning’s careful briefing can fall out of a window that was nowhere near full.
Reading and writing capacity are not the same. A model that accepts a million tokens going in caps a single answer far lower, typically near a tenth of that. It can read a book and cannot write one back in one go.
- A 50-page report, around 20,000 words. Fits anywhere with room to spare.
- A small codebase, around 200,000 words. Fits in the largest current windows.
- A year of company email, several million words. Does not fit, and a bigger window is not the fix.
The window keeps growing. “Fits” is still not the same as “works well”.
Why a bigger window is not automatically better
Section titled “Why a bigger window is not automatically better”Accuracy on the relevant part falls. Models perform worse on long contexts than on short ones. The detail that matters gets buried in the detail that does not, and what comes back is fluent and slightly off rather than obviously wrong, which is the hardest kind of error to catch. This has improved a great deal and has not gone away.
Answers take longer. A full window is more text to get through before the first word comes back. Mildly annoying in a chat, and it compounds in anything that runs in a loop.
Cost rises, though less simply than it used to. Sent cold, a 100,000-token prompt costs roughly 50 times what a 2,000-token one does, and the largest windows carry a price step partway up. But text re-sent unchanged, which is exactly what a fixed document or knowledge base is, is now billed at a fraction of the first-time rate by every major provider. Pushing a big document through every turn is still slower and still less accurate. It is no longer expensive in the way it once was.
A tight, relevant prompt of 2,000 words beats a dumped-everything prompt of 200,000. Almost always, and now for reasons that have little to do with the bill.
What happens when the window fills up
Section titled “What happens when the window fills up”A long session eventually runs past the window. What happens next is a choice made by whoever built the product.
It drops the oldest turns. The model stops seeing the start of the conversation. It is not forgetting; that text simply was not in the block on this turn.
It summarises the older part and keeps the summary. That buys room, and the summary is thinner than what it replaced. What gets compressed away is the specific detail, not the general shape.
It refuses. The request comes back as an error, or a long upload is rejected outright.
From outside, the first two are almost impossible to tell apart, and both feel like an assistant that has quietly stopped following an instruction it was given hours ago. The product rarely says which one it did. You get a slightly worse answer with nothing to tell you the input changed.
Why every new chat starts blank
Section titled “Why every new chat starts blank”A new conversation contains none of the previous one. No memory of yesterday, none of the person typing.
The major chat products ship a separate “memory” layer that stores facts between chats: a small file of notes the product writes, then puts into each new conversation’s block. The model still has no memory. The product is doing the remembering, and what it writes down is a design decision rather than a property of the model.
Where this lands in real systems
Section titled “Where this lands in real systems”Which messages stay, which get summarised, which get dropped, when a fresh window starts, what the summary is told to preserve: those decisions shape the experience more than which model is wired up underneath.
Nobody chose to lose that sentence. Someone chose what the summary was told to keep, and the sentence was not on the list. Deciding what goes into the window is most of the work in building anything that runs for a long time.