The core idea, in one sentence
Retrieve relevant passages from an external corpus → put them in the LLM's context → generate the answer. That's it. The 'augmentation' is the retrieved context; the 'generation' is the LLM's response conditioned on it.
Why RAG instead of fine-tuning?
Fine-tuning teaches a model new behavior. RAG teaches it new facts. If your knowledge changes weekly, RAG wins — re-index, no retrain.
The 5-step RAG pipeline
Every production RAG system, no matter how fancy, fits this pipeline:
- →Ingest — chunk documents and store each chunk in a vector database with its embedding
- →Embed — convert the user query to an embedding using the same model
- →Retrieve — fetch the top-k most similar chunks via approximate nearest-neighbor search
- →Rerank (optional but recommended) — use a smaller, smarter model to re-order the top-k
- →Generate — pass query + top passages into the LLM with a 'use only the context' instruction
Why classic RAG is not enough anymore
Classic RAG retrieves once and hopes. In practice that fails when the query is multi-hop ('compare X and Y'), ambiguous, or when retrieval surfaces irrelevant passages. 2024–2025 saw four important upgrades:
- →CRAG (Corrective RAG) — a critic decides if retrieval was good; if not, falls back to web search
- →GraphRAG — builds a knowledge graph from documents; retrieves by graph traversal; great for multi-hop
- →Adaptive RAG — router decides: no retrieval, single-hop, or multi-hop based on query complexity
- →Multimodal RAG — retrieves text, images, tables together; essential for PDFs with charts
What you actually need in production
A working RAG system needs more than the 5 steps. Real systems have:
- →A good chunking strategy (chunk size silently destroys quality if it's wrong)
- →Hybrid retrieval (vector + keyword + reranker)
- →Evaluation with RAGAs, golden sets, and LLM-as-judge
- →Observability — trace every retrieve + rerank + generate call
- →Citation grounding — show which chunks the answer came from