What is inference in AI?

Inference is using a trained AI model to produce an answer. Every time someone asks an AI tool a question, the model runs inference. Training is the expensive, one-off process of building a model. Inference is every use after that, and it is what AI providers charge for.

Why it matters for a business

The gap between the cost of training and the cost of inference is why businesses rent models rather than build them: someone else pays to build the model, and the business pays per use. That gap is narrower than it was. Reasoning models do much of their work at inference time, and an agent working through a task can make hundreds of calls, so inference is now a line on the monthly bill that teams design around. The same open model can also be run by different inference providers, or on the company’s own machines, at different speeds and prices.

Go deeper

How AI learns: the 4 stages of training a model

On Atlas: LLM serving and inference, compared

Related terms

• Token pricing
• Training data
• Reasoning model
• Private LLM
• Token

All terms, A to Z