What is LLMOps?

LLMOps is the practice of running applications built on large language models reliably once they are in everyday use. It covers tracking which model version and instructions are in use, and testing changes before they go live. It also covers logging what the AI was asked and what it answered, and monitoring cost, speed and quality. The name comes from MLOps, the same practice for traditional machine learning.

Why it matters for a business

AI applications do not hold still after launch the way ordinary software does. Volume grows, prompts get edited, prices move, and providers update or retire models. LLMOps is what makes those changes visible and reversible. For a mid-sized company it need not be a platform: a pinned model version with a fallback behind it, a test set re-run on every change, cost read per request with a spending limit, and logs someone can search cover most of it.

Go deeper

Running AI in production: cost, security, governance and drift

On Atlas: LLM observability and evaluation tools, compared

Related terms

• Model drift
• AI evals
• AI governance
• Token pricing
• Golden dataset

All terms, A to Z