2 RAG-enabled Systems and Their Design
This chapter introduces how Retrieval Augmented Generation is turned into a working system by pairing a prepared knowledge base with real‑time question answering. It explains that RAG-enabled applications hinge on two complementary pipelines: an offline indexing pipeline that consolidates and maintains non‑parametric memory from disparate sources, and an online generation pipeline that retrieves, augments, and uses an LLM to produce contextual answers. The chapter motivates why consolidation is crucial—collecting data in one place, normalizing formats, and splitting it into manageable pieces—so the system can respond accurately and efficiently at runtime.
The indexing pipeline covers connecting to external sources, extracting and parsing content, splitting text into chunks, converting text to embeddings, and storing them in vector databases, with ongoing updates to keep the knowledge base current. The generation pipeline then uses a retriever to find relevant context, prompt management to combine it with the user query, and an LLM setup to generate answers, balancing quality and latency. Around these cores sit production concerns: orchestration to coordinate components, evaluation and monitoring for quality and reliability, and supporting capabilities such as caching for performance, guardrails for policy and safety, and security against threats like prompt injection and data poisoning.
Evaluation is framed as checking alignment across prompt, retrieved context, and final answer, using metrics such as relevance, recall, and precision, often grounded by curated datasets and reinforced with continuous monitoring and user feedback. The chapter also sketches the evolving LLMOps infrastructure that powers such systems across layered services: data and vector storage, model providers, prompt tooling, evaluation frameworks, orchestration, deployment and hosting, and monitoring. By the end, readers gain a coherent blueprint of RAG system design—its pipelines, quality loop, and operational stack—setting the stage for deeper dives into each component in subsequent chapters.
Generation Pipeline covering the five steps of Retrieval Augmented Generation
Indexing Pipeline covering the steps to create the Knowledge Base for RAG
The Indexing Pipeline and the Generation Pipeline together make a RAG-enabled system.
Components of a production ready RAG-enabled system
Four Components of the indexing pipeline facilitate the creation of the knowledge base
Three Components of the generation pipeline enable the real-time query-response process of a RAG enabled system
The triad of RAG evaluation proposed by TruEra
Summary
- A Retrieval Augmented Generation (RAG) enabled system consists of two main pipelines: the Indexing Pipeline and the Generation Pipeline.
- The Indexing Pipeline is responsible for creating and maintaining the knowledge base, which involves data loading, text splitting, data conversion (embeddings), and data storage in a vector database.
- The Generation Pipeline manages real-time interactions by retrieving information, augmenting queries, and generating responses using a Large Language Model (LLM).
- Evaluation and monitoring are crucial components to assess system performance, covering the relevance between the retrieved information and query, the final response and retrieved information, and the final response and the original query.
- The service infrastructure for RAG-enabled systems includes layers for data, models, prompts, evaluation, app orchestration, deployment, application hosting, and monitoring.
- Additional components like caching, guardrails, and security measures are often employed to improve performance, ensure compliance, and address potential threats.
A Simple Guide to Retrieval Augmented Generation ebook for free
