How AI models are trained

Training is not one process. It runs in 4 stages, each doing a different job: learning language and world knowledge, learning to follow a task, learning tone and what to refuse, and learning to work a problem through before answering. The phrases that turn up in every launch post name pieces of it. Trained on “10 trillion tokens” of internet text, books and code is the first stage. “Fine-tuned with human feedback” names the middle two, where people supply the examples and the judgements. “Trained to reason” is the fourth. Each one leaves a mark on how the model behaves at work.

A token is a chunk of text: a short word, or a piece of a longer one. 10 trillion of them is more text than a person could read in a thousand lifetimes. What an AI model actually is covers why models count text that way.

Start with nothing

A model, before it is trained, is a structure and nothing else: a network of nodes and connections, with the connection strengths (called weights) set to random numbers. Ask it to predict the next word and it gives you nonsense.

Training takes those random weights and nudges them, billions of times, until the model becomes useful. (Weights and parameters are two words for the same thing. Both get used, and nothing turns on the difference.)

The basic loop of training

Take a piece of text from the training data and cut off the last word. Show the model what is left and ask what comes next. Compare its answer with the word that was actually there. Then adjust every weight in the network a tiny bit, in the direction that would have made the right word slightly more likely. Then do it again, on a different piece of text.

Do that billions of times across trillions of words and the weights settle into a configuration that captures how language works. That configuration is what you are talking to when you open ChatGPT or Claude.

Example

Let’s say the piece of text is “The cat sat on the mat.” The model is shown “The cat sat on the” and predicts “purple”. The word that was actually there is “mat”, so every weight shifts a little towards making “mat” more likely the next time this shape of sentence comes up. One pass changes almost nothing. Trillions of passes are what build the model.

Four stages, not one

When people say “training”, they usually mean 4 stages that run in order. Each does something different, and the first one is far larger than the other 3.

Pre-training

The model reads trillions of words pulled from the internet, books, papers, code repositories and forums, and plays the next-word game above against all of it.

What comes out is a base model. It is a very good next-word predictor. It can continue almost any sentence, in almost any style, on almost any topic, and it has no idea how to be helpful.

Pre-training is where the model learns language and general knowledge of the world. It is also the expensive step, by a wide margin. Most labs treat the details as commercial information, but Meta published the numbers for its largest openly released Llama run: over 15 trillion tokens of text, on more than 16,000 chips.

Example

Let’s say a base model is shown “How do I make pasta?” It might answer with a recipe. It might also continue with “is a common question among new cooks”, because both are plausible continuations of that line of text. Nothing has yet told it that a question is something to answer.

Instruction tuning

The base model is now shaped to follow instructions. The training data shifts from raw text to prompts paired with good responses: a prompt saying “summarise the following email”, and beside it a summary worth copying.

After thousands of these, the model learns the pattern. When the input looks like a task, produce something that looks like the task fulfilled. This is what makes a chat product feel like an assistant rather than an autocomplete engine.

Alignment

The model produces several possible responses to the same prompt. Human reviewers say which is better. The model is then trained to prefer the kind of response the reviewers liked.

This step sets the model’s tone, polite and hedged and careful, and it sets what the model refuses to do. It is why two labs’ models have noticeably different personalities on similar underlying ability. They were rated by different people against different instructions.

The acronym for it is RLHF, reinforcement learning from human feedback.

Reasoning training

The newest stage, and the only one anyone can watch happen. Ask a current model something involved and the product pauses, shows a thinking indicator, sometimes shows a summary of what it is working through, and answers after a delay. That is a trained behaviour, and it is trained differently from the 3 stages above.

Those 3 all learn from work people produced: text people wrote, answers people demonstrated, responses people ranked. This one learns from problems that can be marked automatically. A maths question with one right answer. A piece of code that either passes its tests or does not. A puzzle with a checkable solution.

The model is handed the problem, produces an attempt, and is scored on whether it landed. Then again, at enormous scale, with the attempts that got there reinforced. The name for training this way is reinforcement learning: learning by trying, being scored and trying again, rather than by copying a worked example.

What comes out of it was not written down by anyone. The labs report the same behaviours turning up regardless: the model catching its own mistakes, breaking a hard step into smaller ones, switching approach when the current one stalls. It finds on its own that generating long working before committing to an answer raises its hit rate. DeepSeek published its version of the method in full, so this stage is better documented from the outside than most of what the labs do.

The working is generated text. It runs through the same loop as the answer, it takes time, and it is billed the same way. Anthropic’s documentation is explicit that thinking tokens are charged as output. A reasoning answer is slower and costs more than a plain one, sometimes several times more.

More thinking also buys accuracy with diminishing returns, so it is worth spending in some places and not others. It earns its cost where a wrong middle step ruins everything after it: a margin calculation across tiered discounts, a delivery note reconciled against a contract, code that has to run. It buys nothing on a question the model either knows or does not, and nothing on drafting an email. In most products this is a setting on the same model rather than a different model, which is why a reasoning model is a mode more than a category.

Training data, and what the model has seen

What a model knows, and how it sounds while saying it, both trace back to the training data.

For the big models, that usually includes:

• Most of the public internet, scraped, deduplicated and filtered
• Millions of books
• Academic papers
• Code repositories, especially open-source
• Forum posts
• Wikipedia, in many languages

What is not in it:

• The company’s internal documents
• Private databases
• Anything behind a paywall, mostly
• Anything that happened after the knowledge cutoff, the date the training data ends

What an AI model actually knows takes that gap apart properly.

Why this costs what it costs

Training one of the biggest models takes weeks on tens of thousands of specialised chips. The cost sits in 3 places:

• The data. Collecting, cleaning, deduplicating and filtering trillions of words is a project in its own right.
• The compute. Running the model across all that data and adjusting weights billions of times takes an enormous amount of power.
• The people. Instruction tuning, alignment and reasoning training all need designed examples, reviewers and problem sets.

The labs do not publish what a run costs. The figures in circulation are outside reconstructions from chip counts and energy use, and the most careful of them, from Epoch AI, has the cost of the largest runs growing by roughly 2.4 times a year, year after year. Treat any single figure as an estimate. What holds is that the cost is far beyond what a business would spend to solve one problem, which is why only a handful of labs train at this level. Some of them release the finished weights afterwards, a separate decision covered in open vs closed models.

Using a trained model, which is called inference, is cheap by comparison. A short request costs a fraction of a cent in compute. That asymmetry is why businesses rent capability rather than build it: someone else pays to build the model, and the business pays per use.

The asymmetry is narrower than it was. A reasoning model spends real compute at question time rather than only at build time, and an agent working through a task makes hundreds of calls rather than one. Inference is still cheap per request measured against the cost of training. It is no longer small enough to ignore measured against a monthly bill, which is why cost per request is now something teams design around rather than assume away.

So the question for a business is almost never “should we train a model?”. It is “how do we get an existing one to do our work?”, and the 3 ways to adapt a model are the subject of prompting, RAG, and fine-tuning.

The model you talk to is a snapshot

A specific model, the one behind whatever chat tab is open right now or whichever version a vendor wired into a product, is the result of one specific training run. Its knowledge cutoff is the date its data ends. Its tone came out of its instruction tuning and alignment. Its quirks, where it refuses and how cautious it gets around medical or legal wording, were set by the choices of the lab that built it.

When a lab releases a new version, it has repeated some or all of those stages, sometimes with the same design and more data, sometimes with a meaningfully different approach. The result is a different model even when the marketing name carries continuity. The change usually shows up as a shift in how answers are shaped rather than as a drop in quality, which is what makes it easy to miss.

Example

Let’s say a nightly job asks a model for the top 5 complaints in yesterday’s support tickets. The reply comes back as 5 lines, and a script turns each line into a row of a report. A new version ships under the same name. The answer is as good as before, and it now arrives as a summary paragraph followed by 5 headed sections. Every line of it becomes a row. The report fills with headings and half-sentences, and nothing in the run errors, so the first person to notice is whoever reads the report.

Which is why production systems care about the exact version they call. The companies selling access to models publish a fixed version label alongside the moving one: a name that points at a single training run and will not change underneath you. Systems that matter call the fixed one. The alternative is the model changing quietly under everything the team built on top of it.

And it is why a model’s oddities are inherited rather than arbitrary. The model that hedges every medical claim, or formats every answer with bullet-pointed headings, behaves that way because the people who shaped it wanted it to. Knowing which model is in play is most of knowing what it will do.