7 Retriever: Returning more relevant results
As agentic systems grow beyond short, self-contained prompts, they run into hard limits on context size, latency, and cost. The chapter explains that an agent cannot realistically “read everything” in a large knowledge source, so it must rely on selective attention: first search cheaply, then spend expensive reasoning only on the most promising results. The Retriever pattern is presented as the practical way to bridge this gap by turning huge external data sources into a small, actionable context that an agent can actually use.
Retrieval is described as a pipeline with three main steps: representing the query in a search-friendly form, executing a broad and inexpensive search, and then refining the candidates with post-processing such as filtering or re-ranking. This structure makes it possible to search millions of items while keeping compute manageable, and it improves grounding by tying outputs to retrieved evidence rather than model guesswork. The chapter emphasizes the tradeoff between recall and precision, showing how a wide first pass can capture likely matches while a later stage removes noise and selects the most relevant items.
The chapter also shows where retrieval fits in real agent designs, especially retrieval-augmented generation, long-term memory, routing, and tool selection. It compares dense semantic search, sparse keyword search, hybrid retrievers, and active resampling approaches, explaining how different variants help in different situations. Through examples like insurance adjudication, technical support, dynamic few-shot example selection, and semantic tool routing, the chapter demonstrates that retrieval helps agents stay lean, accurate, and scalable while still reaching into vast bodies of information when needed.
The AFD of the Retriever pattern. The input (a query) is first processed into a representation suitable for search. This representation is used to query a data storage, producing a set of candidate results. These candidates are optionally passed through a result-processing step (e.g., a re-ranker or filter) to improve the selection before being returned as the final output.
The AFD of a Hybrid Retriever. The query enters the retriever and is processed in two separate streams, each of a different kind. Once processed, two different kinds of searches are performed on the same database. The results of these searches are post-processed separately before being merged together. The merging of the results usually entails concatenation and some naive de-duplication. More sophisticated merging approaches involve IEs that estimate the relevance of the individual results to the query.
The AFD of a Resampling Retriever. The system retrieves data, evaluates its relevance, and branches: either returning the data or re-sampling the search query and attempting to find relevant results again.
Summary
- The “scaling wall of context” and rising computational costs limit an agent’s ability to process vast amounts of data, making brute-force “read everything” approaches expensive and often technically unfeasible.
- The Retriever pattern provides a principled pipeline for selective attention, bridging massive external datasets and the focused, actionable context required for execution.
- Retriever instances typically operate as a multi-stage pipeline: query representation (for example, producing vector embeddings), inexpensive high-recall search execution, and more sophisticated post-processing for precise filtering or re-ranking.
- Retriever provides drastic selectivity and supports semantic alignment by grounding actions in retrieved facts, which improves interpretability, reduces hallucinations, and lowers material costs.
- Hybrid Retrievers fuse semantic and keyword-based searches to balance intent with exact-match precision, and Resampling Retrievers actively refine the search query when initial results lack relevance.
- Across use cases such as automated insurance adjudication, dynamic few-shot exemplar selection, and semantic tool routing, Retriever serves as a foundational gateway that keeps agents lean and fast while enabling access to virtually unlimited knowledge sources.
Agent Design Patterns ebook for free
