What is RAG (retrieval-augmented generation)?

RAG, short for retrieval-augmented generation, is a method where an AI system first searches a set of documents and then writes its answer from what it found. When a question comes in, the system finds the most relevant passages, for example in a company’s policies or manuals. It gives those passages to the AI model along with the question. The model answers from that material instead of from general knowledge, and can point to its sources.

Why it matters for a business

No model was trained on a company’s policies, contracts or product specs, so without retrieval it answers questions about them from the average of other businesses, fluently and wrongly. RAG closes that gap without retraining anything, and a changed document can be searchable the same day. Its weak point is the search. When a RAG system gets an answer wrong, the usual cause is that the wrong passage came back, and the wrong answer looks exactly like a right one, citations included.

Example

Let’s say a support assistant is asked about returning a clearance item. The search brings back the general returns policy instead of the clearance terms. The model quotes the general policy accurately, and the customer is told they have 30 days when they have none.

Go deeper

RAG vs. fine-tuning vs. prompting, and what 'train it on our data' means

On Atlas: RAG frameworks and engines, compared

Related terms

• Grounding
• Embeddings
• Vector database
• Semantic search
• Fine-tuning

All terms, A to Z