11 Correction, planning, and reasoning

Retrieval quality alone is not enough for real-world RAG systems because the core weakness of many pipelines is structural: they retrieve once, generate once, and never check whether the evidence is current, specific, or complete. In high-stakes settings such as compliance, legal research, or medicine, that single-pass design can turn a plausible match into a confident but wrong answer. The chapter argues that RAG must evolve from a passive pipeline into an agentic loop that can evaluate what it retrieved, decide whether it is sufficient, and try again when it is not.

To explain that shift, the chapter contrasts two complementary approaches. CRAG is the corrective approach: it adds a retrieval evaluator that audits the returned documents, filters noisy context, and triggers a fallback web search when the evidence is stale, ambiguous, or irrelevant. LevelRAG is the planning-first approach: it decomposes complex questions into atomic sub-queries, routes them to the most appropriate retrievers, and verifies whether the combined evidence fully answers the original request. CRAG is lightweight and easy to bolt onto existing systems, while LevelRAG requires a broader re-architecture of retrieval control flow.

The chapter ties these ideas to concrete failure modes in an enterprise compliance assistant, showing how staleness, long-tail entity confusion, and multi-hop reasoning gaps map to different architectural fixes. CRAG is best when the main problem is outdated or weak evidence that needs correction after retrieval, while LevelRAG is better when the question itself must be planned and decomposed before searching. In practice, the strongest systems combine both: planning to break the problem apart, correction to audit each sub-result, and verification to decide when the evidence is finally sufficient. The broader lesson is that production RAG is becoming less about a single retrieval step and more about controlled, iterative research.