6 Evaluating Retrieval-Augmented Generation (RAG)

Retrieval-augmented generation is presented as a way to reduce hallucinations by grounding model outputs in external, verifiable documents. The chapter emphasizes that these systems are really two coupled parts: retrieval, which finds candidate evidence, and generation, which turns that evidence into an answer. Because either part can fail on its own, and because their failures can mask one another, evaluation has to separate retrieval quality from generation quality instead of judging only the final response.

To evaluate retrieval, the chapter draws on classic information retrieval metrics such as precision, recall, precision@k, mean average precision, mean reciprocal rank, and nDCG. It explains how exact matching, sparse retrieval, dense retrieval, and hybrid retrieval each have different strengths and failure modes, especially around lexical variation versus precise identifiers. The key lesson is that retrieval evaluation requires labeled corpora, queries, and relevance judgments, and that rank matters because relevant documents buried too low in the context window may not influence the model effectively.

The chapter then introduces a four-part diagnostic framework for RAG systems: faithfulness, answer relevance, context recall, and context precision. Faithfulness checks whether claims in the answer are supported by retrieved context, while answer relevance checks whether the answer actually addresses the user’s question. Context recall asks whether the needed information is present in the retrieved material, and context precision asks whether that material is mostly useful rather than noisy. Taken together, these metrics help diagnose whether failures come from retrieval, generation, query formulation, or agentic planning, and they are especially useful in production through offline evaluation, regression testing, and online monitoring.

The basic single-turn Retrieval-Augmented Generation architecture. A user question is routed to a retrieval system, which queries a knowledge base and returns relevant documents. The LLM synthesizes an answer to the question and retrieves context, producing responses supported by sources.

Agentic RAG. The AI model decides how to query, which retrieval tool to use, and whether to iterate until it has enough evidence to answer.

Deep Research. The AI agent is given a topic to research; it formulates a research plan, searches and refines to gather information, and produces a detailed report.

The generalized RAG framework. The system prompt provides standing instructions. The user prompt provides the query. In-context knowledge, retrieved from external sources, populates the context window. The assistant response is produced from all three inputs combined.

Relationship pairs in RAG evaluation. The diagram labels four evaluation relationships: user query to retrieved context (retrieval relevance), user query to response (answer relevance), retrieved context to response (faithfulness), and response to itself (internal consistency).

Changing the location of relevant information within the language model's input context results in a U-shaped performance curve. Models are better at using relevant information that occurs at the very beginning (primacy bias) or end of their input context (recency bias), and performance degrades significantly when models must access and use information located in the middle of the input context. Source: Liu et al, 2023[2].

• RAG systems separate knowledge from intelligence: the retrieval system provides knowledge; the model provides reasoning. This separation solves the static knowledge problem but introduces two interacting failure modes—retrieval quality and generation quality—that previously covered evaluation methods cannot diagnose.
• Agentic RAG systems add a third failure category: reasoning quality. The model formulates its own retrieval queries, evaluates results, and decides when to stop retrieving. Deep research agents add a planning step and a human verification stage on top of this, each introducing its own evaluation dimension.
• The three retrieval paradigms fail in complementary ways: grep/ripgrep fails on lexical variation; BM25/TF-IDF fails on paraphrase; dense vector search fails on precise identifiers such as SKUs, error codes, and contract clause numbers. Hybrid retrieval using Reciprocal Rank Fusion combines sparse and dense signals to cover both failure modes.
• Precision and recall measure whether the right documents were retrieved. Rank-aware metrics—Precision@k, MRR, MAP, and nDCG—additionally penalize systems that rank relevant documents below the context window cutoff, where they cannot influence generation.
• RAGAS defines four independent metrics across two axes. Faithfulness and answer relevance measure generation quality: faithfulness checks whether claims are supported by retrieved context using claim decomposition; answer relevance checks whether the answer addresses the question using back-translation. Context recall and context precision measure retrieval quality: context recall checks whether the retrieved context covers the ground truth answer; context precision checks whether retrieved documents are relevant and ranked above noise.
• The RAGAS diagnostic matrix maps combinations of high and low metric scores to specific root causes. Low faithfulness with high context scores points to a generation problem—either intrinsic hallucination (misreading the context) or extrinsic hallucination (ignoring it). Low context recall with high faithfulness points to a retrieval gap that should be diagnosed against the retrieval method's known failure mode before any fix is attempted.
• RAGAS generalizes across context source types. SQL retrieval is evaluated with execution accuracy and task success rate; API retrieval requires freshness checks in addition to functional correctness; multi-hop retrieval chains require applying RAGAS at each hop independently, not just at the final output.
• Production RAG evaluation requires three modes: offline evaluation with the full RAGAS suite against a labeled evaluation set; online monitoring of faithfulness and answer relevance for production traffic; and regression testing before every system change. Always evaluate retrieval quality before interpreting generation metrics—poor retrieval degrades generation scores even when the generation model is working correctly.

FAQ