Building an Agentic RAG Application

Step into the role of an AI engineer building a semantic search system for a compliance team drowning in regulatory text. Working with the EU AI Act, a dense, hundred-page policy document, you’ll create a retrieval pipeline that lets analysts ask plain-language questions and instantly surface the most relevant passages by meaning, not just keyword match. You’ll set up a vector database, chunk the text in ways that preserve legal context, generate and store embeddings, and implement metadata-filtered retrieval. Along the way, you’ll validate chunk quality and package your retrieval logic into reusable functions.

Semantic search understands meaning but fumbles exact terms. Keyword search nails specifics but misses nuance. What if you didn’t have to choose? In this liveProject, you’ll supercharge the retrieval system by fusing both into a smarter hybrid search engine. You’ll implement the battle-tested BM25 algorithm, and fuse their rankings with reciprocal rank fusion (RRF) into a single hybrid ranking. Then you'll prove it works by building a test set and measuring precision, recall, MRR, NDCG, and answer-quality metrics like relevance and groundedness across all three strategies.

In this liveProject, you’ll build the generation layer of a document-grounded Q&A system that lets users interrogate the EU AI Act and get back answers with real citations, not hallucinations. Using Pydantic AI with OpenRouter and the free Mistral Small 3.1 model, you’ll craft a system prompt that keeps the LLM strictly on-context, implement smart context injection for top-ranked chunks, and add source attribution so every claim links back to a specific passage.

In this liveProject, you'll step into the shoes of an AI engineer turning a retrieval-and-generation pipeline into an autonomous agent. Building on the retrieval, hybrid search, and grounded-answer components from the earlier projects, you'll design a LangGraph state machine that analyzes each incoming question, routes it, and decides when to retrieve. You'll wrap your hybrid retriever as a tool the agent can call, add a self-evaluation node that scores its own answers, and build an iterative refinement loop with retry and fallback strategies for when the first attempt falls short. Finally, you'll deploy the whole agentic workflow behind a FastAPI endpoint with basic logging and monitoring. By the end, you'll have a modular, agentic RAG system you can adapt to any large document corpus.