table of content

Part 1 Getting started with LLMs

1 Introduction to AI agents and applications

1.1 Building LLM-based applications and agents

1.1.1 LLM-based applications: Summarization and Q&A engines

1.1.2 LLM-based chatbots

1.1.3 AI agents

1.2 Introducing LangChain

1.2.1 LangChain architecture

1.2.2 LangChain’s core object model

1.3 Typical LLM use cases

1.4 How to adapt an LLM to your needs

1.4.1 Prompt engineering

1.4.2 Retrieval-Augmented Generation

1.4.3 Fine-tuning

1.5 Which LLMs to choose

1.6 What you’ll learn from this book

1.7 LLM terminology recap

2 Executing prompts programmatically

2.1 Running prompts programmatically

2.1.1 Setting up the environment for this chapter

2.1.2 Minimal prompt execution

2.2 Running prompts with LangChain

2.3 Prompt templates

2.3.1 Implementing a prompt template with a Python function

2.3.2 Using LangChain’s PromptTemplate

2.4 Prompt types

2.4.1 Text classification

2.4.2 Sentiment analysis

2.4.3 Text summarization

2.4.4 Composing text

2.4.5 Question answering

2.4.6 Reasoning

2.5 Reasoning in detail

2.5.1 One-shot learning

2.5.2 Two-shot learning

2.5.3 Providing steps

2.5.4 Few-shot learning

2.5.5 Implementing few-shot learning with LangChain

2.5.6 Chain of Thought

2.6 Prompt structure

Part 2 Summarization

3 Summarizing text using LangChain

3.1 Summarizing a document bigger than the context window

3.1.1 Chunking the text into Document objects

3.1.2 Split

3.1.3 Map

3.1.4 Reduce

3.1.5 MapReduce combined chain

3.1.6 MapReduce execution

3.2 Summarizing across documents

3.2.1 Creating a list of Document objects

3.2.2 Wikipedia content

3.2.3 File-based content

3.2.4 Creating the Document list

3.2.5 Progressively refining the final summary

3.3 Summarization flowchart

4 Building a research summarization engine

4.1 Overview of a research summarization engine

4.2 Setting up the project

4.3 Implementing the core functionality

4.3.1 Implementing web searching

4.3.2 Implementing web scraping

4.3.3 Instantiating the LLM client

4.3.4 JSON to Python object converter

4.4 Enhancing the architecture with query rewriting

4.5 Prompt engineering

4.5.1 Crafting web search prompts

4.5.2 Crafting summarization prompts

4.5.3 Research report prompt

4.6 Initial implementation

4.6.1 Importing functions and prompt templates

4.6.2 Setting constants and input variables

4.6.3 Instantiating the LLM client

4.6.4 Generating the web searches and collecting the results

4.6.5 Scraping the web results

4.6.6 Summarizing the web results

4.6.7 Generating the research report

4.7 Reimplementing the research summary engine in LCEL

4.7.1 Assistant Instructions chain

4.7.2 Web Searches chain

4.7.3 Search and Summarization chain

4.7.4 Web Research chain

5 Agentic workflows with LangGraph

5.1 Understanding agentic workflows and agents

5.1.1 Workflows

5.1.2 Agents

5.1.3 When to use agent-based architectures

5.1.4 Agent development frameworks

5.2 LangGraph basics

5.3 Moving from LangChain chains to LangGraph

5.4 LangGraph core components

5.4.1 StateGraph structure

5.4.2 State management and typing

5.4.3 Node functions and edge definitions

5.4.4 Entry points and end conditions

5.5 Turning the web research assistant into an AI agent

5.5.1 Original LangChain implementation overview

5.5.2 Identifying components for conversion

5.5.3 Step-by-step transformation process

5.5.4 Code comparison and benefits realized

Part 3 Q&A chatbots

6 RAG fundamentals with ChromaDB

6.1 Semantic search

6.1.1 A basic Q&A chatbot over a single document

6.1.2 A more complex Q&A chatbot over a knowledge base

6.1.3 The RAG design pattern

6.2 Vector stores

6.2.1 What’s a vector store?

6.2.2 How do vector stores work?

6.2.3 Vector libraries vs. vector databases

6.2.4 Most popular vector stores

6.2.5 Storing text and performing a semantic search using Chroma

6.3 Implementing RAG from scratch

6.3.1 Retrieving content from the vector database

6.3.2 Invoking the LLM

6.3.3 Building the chatbot

6.3.4 Recap of RAG terminology

7 Q&A chatbots with LangChain and LangSmith

7.1 LangChain object model for Q&A chatbots

7.1.1 Content ingestion (indexing) stage

7.1.2 Q&A (retrieval and generation) stage

7.2 Vector store content ingestion

7.2.1 Splitting and storing the documents

7.2.2 Removing duplication

7.2.3 Ingesting multiple documents from a folder

7.3 Q&A across stored documents

7.3.1 Querying the vector store directly

7.3.2 Asking a question through a LangChain chain

7.3.3 Completing the RAG chain setup

7.3.4 Follow-up question

7.4 Chatbot memory of message history

7.4.1 Amending the prompt

7.4.2 Updating the chat message history

7.4.3 Feeding the chat history to the RAG chain

7.4.4 Putting everything together

7.5 Tracing execution with LangSmith

Part 4 Advanced RAG

8 Advanced indexing

8.1 Improving RAG accuracy

8.1.1 Content ingestion stage

8.1.2 Question-answering stage

8.2 Advanced document indexing

8.3 Splitting strategy

8.3.1 Splitting strategies

8.3.2 Factors to consider

8.3.3 Choosing the right strategy

8.3.4 Splitting by HTML header

8.4 Embedding strategy

8.4.1 Embedding child chunks with ParentDocumentRetriever

8.4.2 Embedding child chunks with MultiVectorRetriever

8.4.3 Embedding document summaries

8.4.4 Embedding hypothetical questions

8.5 Granular chunk expansion

8.6 Semi-structured content

8.7 Multimodal RAG

9 Question transformations

9.1 Rewrite-Retrieve-Read

9.1.1 Retrieving content using the original user question

9.1.2 Setting up the query rewriter chain

9.1.3 Retrieving content with the rewritten query

9.1.4 Combining everything into a single RAG chain

9.2 Generating multiple queries

9.2.1 Setting up the chain for generating multiple queries

9.2.2 Setting up a custom multi-query retriever

9.2.3 Using a standard MultiQueryRetriever instance

9.3 Step-back question

9.3.1 Setting up the chain to generate a step-back question

9.3.2 Incorporating step-back question generation into the RAG chain

9.4 Hypothetical Document Embeddings (HyDE)

9.4.1 Generating a hypothetical document for the user question

9.4.2 Integrating the HyDE chain into the RAG chain

9.5 Single-step and multi-step decomposition

10 Query generation, routing, and retrieval postprocessing

10.1 Content database query generation

10.2 Self-querying (metadata query enrichment)

10.2.1 Ingestion: Metadata enrichment

10.2.2 Q&A on a metadata-enriched collection

10.3 Generating a structured SQL query

10.3.1 Installing SQLite

10.3.2 Setting up and connecting to the database

10.3.3 Generating SQL queries from natural language

10.3.4 Executing the SQL query

10.4 Generating a semantic SQL query

10.4.1 Standard SQL query

10.4.2 Semantic SQL query

10.4.3 Creating the embeddings

10.4.4 Performing a semantic SQL search

10.4.5 Automating semantic SQL search

10.4.6 Benefits of a semantic SQL search

10.5 Generating queries for a graph database

10.6 Chain routing

10.6.1 Setting up data retrievers

10.6.2 Setting up the query router

10.6.3 Integrating the chain router into a full RAG chain

10.7 Retrieval postprocessing

10.7.1 Similarity postprocessors

10.7.2 Keyword postprocessors

10.7.3 Time weighting

10.7.4 RAG fusion (Reciprocal Rank Fusion)

Part 5 AI agents

11 Building tool-based agents with LangGraph

11.1 Starting simple: Building a single-tool travel info agent

11.1.1 Project setup

11.1.2 Loading environment variables

11.1.3 Preparing the travel information vector store

11.2 Enabling agents to call tools

11.2.1 From function calling to tool calling

11.2.2 How tool calling works with LLMs

11.2.3 Registering tools with the LLM

11.2.4 Agent state: Tracking the conversation

11.2.5 Executing tool calls

11.2.6 The LLM node: Coordinating reasoning and action

11.3 Assembling the agent graph

11.4 Understanding the agent graph structure

11.5 Running the agent chatbot: The Read-Eval-Print Loop

11.6 Executing a request

11.6.1 Step-by-step debugging

11.7 Expanding your agent: Adding a weather forecast tool

11.7.1 Implementing a mock weather service

11.7.2 Creating the weather forecast tool

11.7.3 Updating the agent for multi-tool support

11.8 Executing the multi-tool agent

11.8.1 Running the multi-tool agent (initial behavior)

11.8.2 Improving LLM tool usage with system guidance

11.9 Using prebuilt components for rapid development

11.9.1 Refactoring to use the LangGraph ReAct agent

11.9.2 Running the prebuilt agent

11.9.3 Observing and debugging with LangSmith

11.9.4 Enabling LangSmith tracing

12 Multi-agent systems

12.1 Building an accommodation booking agent

12.1.1 Hotel booking tool

12.1.2 B&B booking tool

12.1.3 ReAct accommodation booking agent

12.2 Building a router-based travel assistant

12.2.1 Designing the router agent

12.2.2 Routing logic

12.2.3 Building the multi-agent graph

12.2.4 Trying out the router agent

12.3 Handling multi-agent requests with a Supervisor component

12.3.1 The Supervisor pattern: An agent of agents

12.3.2 From “one-way” to “return ticket” interactions

12.3.3 Trying out the Supervisor agent

13 Building and consuming MCP servers

13.1 Introduction to MCP servers

13.1.1 The problem: Context integration at scale

13.1.2 The solution: The Model Context Protocol

13.1.3 The MCP ecosystem

13.2 How to build MCP servers

13.2.1 Essential resources for MCP server development

13.2.2 Official language-specific MCP SDKs

13.2.3 Consuming MCP servers in LLM applications and agents

13.3 Building a weather MCP server

13.3.1 Implementing the MCP server

13.3.2 Trying out the MCP server with MCP Inspector

13.3.3 Consuming the MCP server from a test MCP host

13.4 Integrating the Weather MCP tool into an agent

13.4.1 Preparing the travel agent for live weather data

13.4.2 Integrating the AccuWeather MCP tool

13.4.3 Updating the agent chat loop

13.4.4 Combining local and remote tools

13.4.5 Testing and verification

13.4.6 Using the agent for complex queries

14 Productionizing AI agents: Memory, guardrails, and beyond

14.1 Memory

14.1.1 Types of memory

14.1.2 Why short-term memory is needed

14.1.3 Checkpoints in LangGraph

14.1.4 Adding short-term memory to our travel assistant

14.1.5 Executing the checkpointer-enabled assistant

14.1.6 Rewinding the state to a past checkpoint

14.2 Guardrails

14.2.1 Implementing guardrails to reject nontravel-related questions

14.2.2 Implementing more restrictive guardrails at the agent level

14.3 Beyond this chapter

14.3.1 Long-term user and application memory

14.3.2 Human-in-the-loop

14.3.3 Post-model guardrails

14.3.4 Evaluation of AI agents and applications

14.3.5 Deployment on the LangGraph platform and Open Agent Platform

Appendices

Appendix A: Trying out LangChain

A.1 Trying out LangChain in a Jupyter Notebook environment

A.1.1 Sentence completion example

A.1.2 Prompt engineering examples

A.1.3 Creating chains and executing them with LCEL

Appendix B: Setting up a Jupyter Notebook environment

B.1 Installing the Python interpreter or a Python distribution

B.2 Creating a virtual environment with venv and upgrading pip

B.3 Setting up a Jupyter Notebook

Appendix C: Choosing an LLM

C.1 Popular large language models

C.1.1 OpenAI GPT series

C.1.2 Gemini

C.1.3 Gemma

C.1.4 Claude

C.1.5 Cohere

C.1.6 Llama

C.1.7 Falcon

C.1.8 Mistral

C.1.9 Qwen

C.1.10 Grok

C.1.11 Phi

C.1.12 DeepSeek

C.2 How to choose a model

C.2.1 Model purpose

C.2.2 Proprietary vs. open source

C.2.3 Model size (number of parameters)

C.2.4 Context window size

C.2.5 Multilingual support

C.2.6 Accuracy vs. speed

C.2.7 Cost and hardware requirements

C.2.8 Task suitability (standard benchmarks)

C.2.9 Safety and Bias

C.2.10 A practical example

C.3 A word of caution

Appendix D: Installing SQLite on Windows

D.1 Installing SQLite

Appendix E: Open source LLMs

E.1 Benefits of open source LLMs

E.1.1 Transparency

E.1.2 Privacy

E.1.3 Community driven

E.1.4 Cost savings

E.2 Popular open source LLMs

E.3 Considerations on running open source LLMs locally

E.3.1 Limitations of consumer hardware

E.3.2 Quantization

E.3.3 OpenAI REST API compatibility

E.4 Local inference engines

E.4.1 llama.cpp

E.4.2 Ollama

E.4.3 vLLM

E.4.4 llamafile

E.4.5 LM Studio

E.4.6 LocalAI

E.4.7 GPT4All

E.4.8 Comparing local inference engines

E.4.9 Choosing a local inference engine

E.5 Inference via the Hugging Face Transformers library

E.5.1 Hugging Face Transformers library

E.5.2 LangChain’s Hugging Face pipeline

E.6 Building a local summarization engine

E.6.1 Choosing the inference engine

E.6.2 Starting up the OpenAI-compatible server

E.6.3 Modifying the original solution

E.6.4 Running the summarization engine through the local LLM

E.6.5 Comparison between OpenAI and local LLM

Overview

4 Building a research summarization engine

This chapter walks through building an end-to-end research summarization engine that turns an open-ended question into a structured report. The system automates the manual workflow of searching the web, reading sources, extracting key points, and synthesizing findings. It starts from a simple, sequential prototype and evolves into a robust, chain-based solution using LangChain and its Expression Language (LCEL), highlighting an architecture that includes query rewriting, web search, page scraping, per-page summarization, and final report generation. Along the way, it emphasizes clarity of prompts, modularity, and practical considerations like performance, deployment options, and cost awareness.

After a brief project setup in a Python environment, the chapter implements core capabilities: a web search function (via a DuckDuckGo wrapper), a web scraper using requests and Beautiful Soup, an OpenAI chat model client initializer, and a small JSON-to-object utility. Prompt engineering underpins the system’s accuracy and structure. A few-shot assistant selector returns a JSON blob describing the most relevant expert persona and its instructions; a web-search prompt rewrites the user’s question into multiple targeted queries to broaden coverage and disambiguate intent; a page-summary prompt extracts concise, factual answers or summaries from scraped text; and a final report prompt synthesizes all summaries into a detailed, well-structured response. Configuration constants (for number of queries, results per query, and text truncation) let developers balance depth, runtime, and token costs.

The initial sequential engine follows a clear flow: select the assistant, generate multiple queries, fetch result URLs, scrape each page, summarize each page with respect to its query, merge the summaries, and produce the final report. To address latency and scale, the chapter then reimplements the pipeline with LCEL as a set of composable mini-chains: Assistant Instructions, Web Searches, Search and Summarization (itself three sub-chains for URLs, text-and-summary, and merged summaries), and a final Research Report chain. Using LCEL operators—such as RunnableLambda, RunnableParallel, StrOutputParser, and map()—the design parallelizes work at two levels: across search queries and across URLs within each query. The result is a faster, cleaner, and more maintainable research engine inspired by established open-source patterns, and ready for experimentation or extension.

Semi-automated summarization with LLM: You prompt ChatGPT to summarize each web search result and then compile them into a consolidated summary.

Automated research summarization engine: Ask a question, and the engine performs a web search, returns URLs, scrapes and summarizes web pages, and compiles a research report for you.

Automated research summarization engine: Ask a question, and the engine performs a web search, returns URLs, scrapes and summarizes web pages, and compiles a research report for you.

Revised system architecture diagram, incorporating query rewriting: The process begins by tasking the LLM to generate a specified number of queries based on the user's research question. These queries are then submitted to the search engine. The subsequent processing remains consistent with the illustration in figure 4.2.

Architecture of the chain-based research summarization engine: Each step of the process is re-implemented as a mini-chain; all mini-chains are assembled into a master Web Research chain.

The Search and Summarization chain is made up of three chains: a web search chain, a scraping and summarization chain, and a summary compilation chain.

Chain architectural diagram showing how parallelization is applied in the research summarization engine, highlighting the execution of separate chain instances in parallel for efficiency.

Enhanced Search and Summarization chain diagram: The Search Result URLs chain generates multiple URLs; each URL initiates an instance of the Result Text and Summary chain, which in turn produces a corresponding summary. These summaries are then consolidated into a single text string by the Joined Summary chain.

Web Research chain: This illustrates how each web search initiated by the Web Searches chain triggers a separate instance of the Search and Summarization chain. These instances operate concurrently, each producing a summary. These summaries are then compiled by the Research Report chain into the comprehensive final research report.

Summary

A research summarization engine is built around an LLM for efficient processing.
The engine's workflow involves:

Receiving a research question from the user.
Generating web searches to gather relevant information with LLM assistance.
Summarizing the content from each search result using the LLM.
Combining these summaries into a detailed report.

This engine can be developed using Visual Studio Code.
Key components of the engine include web searching, web scraping, and LLM interaction.
Prompt engineering is critical in three areas:

Crafting web searches from the user's question.
Summarizing search results.
Creating the final report.

LangChain Expression Language (LCEL) helps organize the workflow into a main chain with secondary chains.
Various techniques can parallelize the execution of these secondary chains for efficiency.

FAQ

What is a research summarization engine and what problems does it solve?

An LLM-powered application that takes a user question, performs web searches, scrapes relevant pages, summarizes each, and compiles a final report. It automates what you’d normally do manually: search, read, take notes, and synthesize findings, improving speed and consistency for a wide range of research tasks.

What are the core components I need to implement first?

Three essentials: a web searching function (DuckDuckGo wrapper in LangChain), a web scraping function (requests + BeautifulSoup) to extract page text, and an LLM client initializer (e.g., ChatOpenAI via langchain_openai). A small utility to safely parse JSON from model outputs into Python objects (to_obj) is also used frequently.

How does the web search integration work in this chapter?

It uses LangChain’s DuckDuckGoSearchAPIWrapper to run a query and return result URLs from the link field. You control the number of results per query and later flatten and pair each URL with its originating search query for downstream scraping and summarization.

How is web scraping implemented and how do we handle noisy content?

Scraping is done with requests and BeautifulSoup, sending a realistic User-Agent header and a timeout. The raw page text can be noisy (menus, navigation, etc.), but the summarization prompt is designed to extract relevant facts from that noise, so heavy pre-cleaning isn’t required for the basic version.

What is “query rewriting” and why is it added to the architecture?

Query rewriting (multiple-query generation) uses the LLM to expand or clarify the original question into several targeted web search queries. It improves coverage and relevance, disambiguates short or vague questions, and can decompose complex topics into simpler searches—resulting in better sources and higher-quality summaries.

How are prompts structured for this engine?

Four prompt types: assistant selection (chooses a persona and returns JSON with assistant_type, assistant_instructions, user_question), web search generation (produces multiple queries as JSON), per-page summarization (answers the search query from scraped text or summarizes if not answerable), and final research report (synthesizes all summaries into a long, structured, source-cited report).

Why return JSON from the LLM and how is it parsed?

JSON enforces predictable, machine-readable outputs that are easy to route into subsequent steps. The chapter uses a lightweight to_obj helper (json.loads with fallback) and, in LCEL chains, StrOutputParser to extract the content string before conversion to dicts/lists.

How does the initial sequential implementation work end-to-end?

Flow: select assistant and instructions → generate multiple search queries → fetch result URLs for each query → scrape each URL → summarize each page → merge summaries and sources → produce the final report. It’s straightforward to understand and debug but slower because every step runs in sequence.

What is LCEL and how does it speed up the engine?

LCEL (LangChain Expression Language) composes pipelines from runnables and enables parallelism, streaming, and clean composition. The chapter builds mini-chains (assistant instructions, web searches, search-and-summarization) and a master Web Research chain, using RunnableLambda, RunnableParallel, and the map operator to parallelize per-query and per-URL work for significant speed gains.

How do I set up, configure, and control cost?

Set up a Python project (VS Code, virtual environment), install packages (langchain, langchain_openai, requests, bs4, duckduckgo-search, etc.), and put your OpenAI API key in a .env file. Control cost and latency by tuning NUM_SEARCH_QUERIES, results per query, and max characters per page; more pages and tokens yield better coverage but higher cost and longer runtime. You can start with a CLI script and later expose the chain as a REST API if desired.

pro $24.99 per month

access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
choose one free eBook per month to keep
exclusive 50% discount on all purchases
renews monthly, pause or cancel renewal anytime

lite $19.99 per month

access to all Manning books, including MEAPs!

team

5, 10 or 20 seats+ for your team - learn more

eBook

pdf, ePub, online

$47.99 $23.99

you save $24.00 (50%)

include audio $24.99 $12.49

pro $24.99 per month

access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
choose one free eBook per month to keep
exclusive 50% discount on all purchases
renews monthly, pause or cancel renewal anytime

lite $19.99 per month

access to all Manning books, including MEAPs!

team

5, 10 or 20 seats+ for your team - learn more

eBook

$47.99 $23.99

you save $24.00 (50%)

include audio $24.99 $12.49

eBook

pdf, ePub, online

$47.99 $23.99

you save $24.00 (50%)

include audio $24.99 $12.49

pro $24.99 per month

access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
choose one free eBook per month to keep
exclusive 50% discount on all purchases
renews monthly, pause or cancel renewal anytime

lite $19.99 per month

access to all Manning books, including MEAPs!

team

5, 10 or 20 seats+ for your team - learn more