What is AI bias?

AI bias is when an AI system’s results are consistently unfair to certain groups of people. It usually comes from the data the AI learned from. A hiring tool trained on past hiring decisions can repeat the preferences in them, and a language model can repeat stereotypes found in the text it learned from.

Why it matters for a business

Bias is invisible in any single answer, which reads as considered and confident. It only shows when outputs are compared across groups, and only if someone looks. Removing the protected field, such as gender or age, does not remove it, because postcode, university, employment gaps and phrasing carry much of the same signal. Where AI shapes decisions about people, in hiring, lending or pricing, the controls are review across groups and a person in the loop, and some places now regulate it: New York City requires an annual independent bias audit of automated hiring tools.

Example

Let’s say a screening model learns from 5 years of hires, most of whom came from a handful of universities. It does not learn that this was a preference. It learns it as what a good candidate looks like, and ranks the next applicant from elsewhere lower.

Go deeper

The 4 risks of using AI in business

Related terms

• Training data
• Explainable AI
• Responsible AI
• Human in the loop
• AI governance

All terms, A to Z