Glossary

What are AI benchmarks?

AI benchmarks are standard tests used to measure and compare AI models on the same tasks, such as coding problems, maths questions or exam papers. AI labs publish their models’ benchmark scores at launch, and public leaderboards rank models by them. A score shows how a model did on that test, not how it will do on a particular company’s work.

Why it matters for a business

A benchmark score is a narrow claim. The rankings change every few months as new models ship. The tasks are generic and may look nothing like a particular company’s work. Test questions can leak into the material a model was trained on, which inflates its score. And benchmarks miss much of what decides daily use: how a model phrases things, handles ambiguity and follows instructions. A small test on a team’s own past cases says more about whether a model fits the job than any leaderboard.