table of content

Part 1 Getting started with LLMs

1 Introduction to AI agents and applications

1.1 Building LLM-based applications and agents

1.1.1 LLM-based applications: Summarization and Q&A engines

1.1.2 LLM-based chatbots

1.1.3 AI agents

1.2 Introducing LangChain

1.2.1 LangChain architecture

1.2.2 LangChain’s core object model

1.3 Typical LLM use cases

1.4 How to adapt an LLM to your needs

1.4.1 Prompt engineering

1.4.2 Retrieval-Augmented Generation

1.4.3 Fine-tuning

1.5 Which LLMs to choose

1.6 What you’ll learn from this book

1.7 LLM terminology recap

2 Executing prompts programmatically

2.1 Running prompts programmatically

2.1.1 Setting up the environment for this chapter

2.1.2 Minimal prompt execution

2.2 Running prompts with LangChain

2.3 Prompt templates

2.3.1 Implementing a prompt template with a Python function

2.3.2 Using LangChain’s PromptTemplate

2.4 Prompt types

2.4.1 Text classification

2.4.2 Sentiment analysis

2.4.3 Text summarization

2.4.4 Composing text

2.4.5 Question answering

2.4.6 Reasoning

2.5 Reasoning in detail

2.5.1 One-shot learning

2.5.2 Two-shot learning

2.5.3 Providing steps

2.5.4 Few-shot learning

2.5.5 Implementing few-shot learning with LangChain

2.5.6 Chain of Thought

2.6 Prompt structure

Part 2 Summarization

3 Summarizing text using LangChain

3.1 Summarizing a document bigger than the context window

3.1.1 Chunking the text into Document objects

3.1.2 Split

3.1.3 Map

3.1.4 Reduce

3.1.5 MapReduce combined chain

3.1.6 MapReduce execution

3.2 Summarizing across documents

3.2.1 Creating a list of Document objects

3.2.2 Wikipedia content

3.2.3 File-based content

3.2.4 Creating the Document list

3.2.5 Progressively refining the final summary

3.3 Summarization flowchart

4 Building a research summarization engine

4.1 Overview of a research summarization engine

4.2 Setting up the project

4.3 Implementing the core functionality

4.3.1 Implementing web searching

4.3.2 Implementing web scraping

4.3.3 Instantiating the LLM client

4.3.4 JSON to Python object converter

4.4 Enhancing the architecture with query rewriting

4.5 Prompt engineering

4.5.1 Crafting web search prompts

4.5.2 Crafting summarization prompts

4.5.3 Research report prompt

4.6 Initial implementation

4.6.1 Importing functions and prompt templates

4.6.2 Setting constants and input variables

4.6.3 Instantiating the LLM client

4.6.4 Generating the web searches and collecting the results

4.6.5 Scraping the web results

4.6.6 Summarizing the web results

4.6.7 Generating the research report

4.7 Reimplementing the research summary engine in LCEL

4.7.1 Assistant Instructions chain

4.7.2 Web Searches chain

4.7.3 Search and Summarization chain

4.7.4 Web Research chain

5 Agentic workflows with LangGraph

5.1 Understanding agentic workflows and agents

5.1.1 Workflows

5.1.2 Agents

5.1.3 When to use agent-based architectures

5.1.4 Agent development frameworks

5.2 LangGraph basics

5.3 Moving from LangChain chains to LangGraph

5.4 LangGraph core components

5.4.1 StateGraph structure

5.4.2 State management and typing

5.4.3 Node functions and edge definitions

5.4.4 Entry points and end conditions

5.5 Turning the web research assistant into an AI agent

5.5.1 Original LangChain implementation overview

5.5.2 Identifying components for conversion

5.5.3 Step-by-step transformation process

5.5.4 Code comparison and benefits realized

Part 3 Q&A chatbots

6 RAG fundamentals with ChromaDB

6.1 Semantic search

6.1.1 A basic Q&A chatbot over a single document

6.1.2 A more complex Q&A chatbot over a knowledge base

6.1.3 The RAG design pattern

6.2 Vector stores

6.2.1 What’s a vector store?

6.2.2 How do vector stores work?

6.2.3 Vector libraries vs. vector databases

6.2.4 Most popular vector stores

6.2.5 Storing text and performing a semantic search using Chroma

6.3 Implementing RAG from scratch

6.3.1 Retrieving content from the vector database

6.3.2 Invoking the LLM

6.3.3 Building the chatbot

6.3.4 Recap of RAG terminology

7 Q&A chatbots with LangChain and LangSmith

7.1 LangChain object model for Q&A chatbots

7.1.1 Content ingestion (indexing) stage

7.1.2 Q&A (retrieval and generation) stage

7.2 Vector store content ingestion

7.2.1 Splitting and storing the documents

7.2.2 Removing duplication

7.2.3 Ingesting multiple documents from a folder

7.3 Q&A across stored documents

7.3.1 Querying the vector store directly

7.3.2 Asking a question through a LangChain chain

7.3.3 Completing the RAG chain setup

7.3.4 Follow-up question

7.4 Chatbot memory of message history

7.4.1 Amending the prompt

7.4.2 Updating the chat message history

7.4.3 Feeding the chat history to the RAG chain

7.4.4 Putting everything together

7.5 Tracing execution with LangSmith

Part 4 Advanced RAG

8 Advanced indexing

8.1 Improving RAG accuracy

8.1.1 Content ingestion stage

8.1.2 Question-answering stage

8.2 Advanced document indexing

8.3 Splitting strategy

8.3.1 Splitting strategies

8.3.2 Factors to consider

8.3.3 Choosing the right strategy

8.3.4 Splitting by HTML header

8.4 Embedding strategy

8.4.1 Embedding child chunks with ParentDocumentRetriever

8.4.2 Embedding child chunks with MultiVectorRetriever

8.4.3 Embedding document summaries

8.4.4 Embedding hypothetical questions

8.5 Granular chunk expansion

8.6 Semi-structured content

8.7 Multimodal RAG

9 Question transformations

9.1 Rewrite-Retrieve-Read

9.1.1 Retrieving content using the original user question

9.1.2 Setting up the query rewriter chain

9.1.3 Retrieving content with the rewritten query

9.1.4 Combining everything into a single RAG chain

9.2 Generating multiple queries

9.2.1 Setting up the chain for generating multiple queries

9.2.2 Setting up a custom multi-query retriever

9.2.3 Using a standard MultiQueryRetriever instance

9.3 Step-back question

9.3.1 Setting up the chain to generate a step-back question

9.3.2 Incorporating step-back question generation into the RAG chain

9.4 Hypothetical Document Embeddings (HyDE)

9.4.1 Generating a hypothetical document for the user question

9.4.2 Integrating the HyDE chain into the RAG chain

9.5 Single-step and multi-step decomposition

10 Query generation, routing, and retrieval postprocessing

10.1 Content database query generation

10.2 Self-querying (metadata query enrichment)

10.2.1 Ingestion: Metadata enrichment

10.2.2 Q&A on a metadata-enriched collection

10.3 Generating a structured SQL query

10.3.1 Installing SQLite

10.3.2 Setting up and connecting to the database

10.3.3 Generating SQL queries from natural language

10.3.4 Executing the SQL query

10.4 Generating a semantic SQL query

10.4.1 Standard SQL query

10.4.2 Semantic SQL query

10.4.3 Creating the embeddings

10.4.4 Performing a semantic SQL search

10.4.5 Automating semantic SQL search

10.4.6 Benefits of a semantic SQL search

10.5 Generating queries for a graph database

10.6 Chain routing

10.6.1 Setting up data retrievers

10.6.2 Setting up the query router

10.6.3 Integrating the chain router into a full RAG chain

10.7 Retrieval postprocessing

10.7.1 Similarity postprocessors

10.7.2 Keyword postprocessors

10.7.3 Time weighting

10.7.4 RAG fusion (Reciprocal Rank Fusion)

Part 5 AI agents

11 Building tool-based agents with LangGraph

11.1 Starting simple: Building a single-tool travel info agent

11.1.1 Project setup

11.1.2 Loading environment variables

11.1.3 Preparing the travel information vector store

11.2 Enabling agents to call tools

11.2.1 From function calling to tool calling

11.2.2 How tool calling works with LLMs

11.2.3 Registering tools with the LLM

11.2.4 Agent state: Tracking the conversation

11.2.5 Executing tool calls

11.2.6 The LLM node: Coordinating reasoning and action

11.3 Assembling the agent graph

11.4 Understanding the agent graph structure

11.5 Running the agent chatbot: The Read-Eval-Print Loop

11.6 Executing a request

11.6.1 Step-by-step debugging

11.7 Expanding your agent: Adding a weather forecast tool

11.7.1 Implementing a mock weather service

11.7.2 Creating the weather forecast tool

11.7.3 Updating the agent for multi-tool support

11.8 Executing the multi-tool agent

11.8.1 Running the multi-tool agent (initial behavior)

11.8.2 Improving LLM tool usage with system guidance

11.9 Using prebuilt components for rapid development

11.9.1 Refactoring to use the LangGraph ReAct agent

11.9.2 Running the prebuilt agent

11.9.3 Observing and debugging with LangSmith

11.9.4 Enabling LangSmith tracing

12 Multi-agent systems

12.1 Building an accommodation booking agent

12.1.1 Hotel booking tool

12.1.2 B&B booking tool

12.1.3 ReAct accommodation booking agent

12.2 Building a router-based travel assistant

12.2.1 Designing the router agent

12.2.2 Routing logic

12.2.3 Building the multi-agent graph

12.2.4 Trying out the router agent

12.3 Handling multi-agent requests with a Supervisor component

12.3.1 The Supervisor pattern: An agent of agents

12.3.2 From “one-way” to “return ticket” interactions

12.3.3 Trying out the Supervisor agent

13 Building and consuming MCP servers

13.1 Introduction to MCP servers

13.1.1 The problem: Context integration at scale

13.1.2 The solution: The Model Context Protocol

13.1.3 The MCP ecosystem

13.2 How to build MCP servers

13.2.1 Essential resources for MCP server development

13.2.2 Official language-specific MCP SDKs

13.2.3 Consuming MCP servers in LLM applications and agents

13.3 Building a weather MCP server

13.3.1 Implementing the MCP server

13.3.2 Trying out the MCP server with MCP Inspector

13.3.3 Consuming the MCP server from a test MCP host

13.4 Integrating the Weather MCP tool into an agent

13.4.1 Preparing the travel agent for live weather data

13.4.2 Integrating the AccuWeather MCP tool

13.4.3 Updating the agent chat loop

13.4.4 Combining local and remote tools

13.4.5 Testing and verification

13.4.6 Using the agent for complex queries

14 Productionizing AI agents: Memory, guardrails, and beyond

14.1 Memory

14.1.1 Types of memory

14.1.2 Why short-term memory is needed

14.1.3 Checkpoints in LangGraph

14.1.4 Adding short-term memory to our travel assistant

14.1.5 Executing the checkpointer-enabled assistant

14.1.6 Rewinding the state to a past checkpoint

14.2 Guardrails

14.2.1 Implementing guardrails to reject nontravel-related questions

14.2.2 Implementing more restrictive guardrails at the agent level

14.3 Beyond this chapter

14.3.1 Long-term user and application memory

14.3.2 Human-in-the-loop

14.3.3 Post-model guardrails

14.3.4 Evaluation of AI agents and applications

14.3.5 Deployment on the LangGraph platform and Open Agent Platform

Appendices

Appendix A: Trying out LangChain

A.1 Trying out LangChain in a Jupyter Notebook environment

A.1.1 Sentence completion example

A.1.2 Prompt engineering examples

A.1.3 Creating chains and executing them with LCEL

Appendix B: Setting up a Jupyter Notebook environment

B.1 Installing the Python interpreter or a Python distribution

B.2 Creating a virtual environment with venv and upgrading pip

B.3 Setting up a Jupyter Notebook

Appendix C: Choosing an LLM

C.1 Popular large language models

C.1.1 OpenAI GPT series

C.1.2 Gemini

C.1.3 Gemma

C.1.4 Claude

C.1.5 Cohere

C.1.6 Llama

C.1.7 Falcon

C.1.8 Mistral

C.1.9 Qwen

C.1.10 Grok

C.1.11 Phi

C.1.12 DeepSeek

C.2 How to choose a model

C.2.1 Model purpose

C.2.2 Proprietary vs. open source

C.2.3 Model size (number of parameters)

C.2.4 Context window size

C.2.5 Multilingual support

C.2.6 Accuracy vs. speed

C.2.7 Cost and hardware requirements

C.2.8 Task suitability (standard benchmarks)

C.2.9 Safety and Bias

C.2.10 A practical example

C.3 A word of caution

Appendix D: Installing SQLite on Windows

D.1 Installing SQLite

Appendix E: Open source LLMs

E.1 Benefits of open source LLMs

E.1.1 Transparency

E.1.2 Privacy

E.1.3 Community driven

E.1.4 Cost savings

E.2 Popular open source LLMs

E.3 Considerations on running open source LLMs locally

E.3.1 Limitations of consumer hardware

E.3.2 Quantization

E.3.3 OpenAI REST API compatibility

E.4 Local inference engines

E.4.1 llama.cpp

E.4.2 Ollama

E.4.3 vLLM

E.4.4 llamafile

E.4.5 LM Studio

E.4.6 LocalAI

E.4.7 GPT4All

E.4.8 Comparing local inference engines

E.4.9 Choosing a local inference engine

E.5 Inference via the Hugging Face Transformers library

E.5.1 Hugging Face Transformers library

E.5.2 LangChain’s Hugging Face pipeline

E.6 Building a local summarization engine

E.6.1 Choosing the inference engine

E.6.2 Starting up the OpenAI-compatible server

E.6.3 Modifying the original solution

E.6.4 Running the summarization engine through the local LLM

E.6.5 Comparison between OpenAI and local LLM

Overview

13 Building and consuming MCP servers

Integrating reliable external context into AI agents has traditionally required teams to build and maintain bespoke wrappers for every data source, leading to duplicated effort and inconsistent results. The Model Context Protocol (MCP) addresses this by standardizing how services expose tools via MCP servers, which agents (MCP hosts) can discover and invoke through MCP clients as if they were local tools. The chapter explains this client–server architecture, how tools are surfaced and selected, and how MCP supports both local and remote transports. With rapid adoption across major LLM platforms and a growing library of third-party servers, MCP is becoming a foundation for scalable, reusable, and interoperable tool ecosystems.

The chapter then turns to practical construction and testing. It recommends building on official SDKs rather than implementing the protocol from scratch, highlighting language options and the evolution from FastMCP 1 to the more capable FastMCP 2 (with improvements in deployment, security, connectivity, and testing). A hands-on example walks through creating a weather MCP server backed by the AccuWeather API, exposing a single tool for current conditions over Streamable HTTP. The server is validated first with MCP Inspector for interactive exploration and then with a lightweight test host, confirming standard tool schemas and result packaging that align with established tool-calling patterns.

Finally, the chapter shows how to consume MCP tools in real applications. It outlines built-in support in popular LLM APIs for executing public MCP tools and covers enterprise scenarios where private servers are accessed via the FastMCP client or frameworks like LangChain’s MultiServerMCPClient. A travel assistant agent is upgraded by swapping a mock weather tool for the remote MCP tool, updating the chat loop for asynchronous calls, and combining remote and local tools in one coherent toolset. The result is an agent that can answer complex, context-rich queries with real-time data—illustrating best practices for configuration, authorization, and reliability, and underscoring why MCP server skills are becoming essential for modern AI development.

MCP host process connecting to multiple local and remote MCP servers via MCP clients, each exposing tools backed by various resources. This architecture enables agents to flexibly access both local documents and online services through standardized tool interfaces

Screenshot of MCP Inspector after installation.

Screenshot of MCP Inspector after connecting to the Weather MCP server.

Screenshot of MCP Inspector after selecting the Tools tab and clicking List Tools.The get_weather_conditions tool is listed, and a corresponding panel is shown on the right after clicking the tool name.

Screenshot of MCP Inspector after entering a location and clicking Run Tool.The output shows the current weather conditions for Penzance, UK, confirming that the tool works as expected.

The travel information agent enhanced with a real weather tool from a remote MCP server, replacing the previous local mock weather tool

Summary

MCP streamlines agent development by shifting tool integration to data/service providers, enabling developers to plug into ready-made tools without writing repetitive wrappers.
The MCP protocol standardizes tool exposure using a flexible client-server model, making it easy for agents (“hosts”) to discover and use tools from one or many MCP servers.
Official and community MCP SDKs—especially FastMCP 2 for Python—make it practical to build, test, and deploy production-grade MCP servers for any use case.
MCP Inspector accelerates development by letting you interactively browse, test, and debug tools exposed by MCP servers without writing a line of client code.
Leading LLM providers like OpenAI now support MCP tools natively, further reducing integration effort for both developers and service providers.
Integration with agent frameworks (e.g., LangChain/LangGraph) allows seamless mixing of local and remote tools, enabling richer, more capable AI assistants.
Mastering MCP is fast becoming essential for anyone building advanced, context-rich AI agents and services in the modern AI ecosystem.

FAQ

What problem does the Model Context Protocol (MCP) solve?

MCP standardizes how data and services are exposed as tools, eliminating the need for every team to build custom wrappers for each API. This reduces duplicated effort, makes integrations consistent, and lets agents discover and invoke remote tools as easily as local ones.

How does the MCP architecture work (host, client, server, tools)?

- The agent app acts as the MCP host process
- It connects to one or more MCP servers via an MCP client
- Each MCP server exposes a set of tools (and optionally prompts/resources)
- The host discovers and calls only the tools it needs, similar to consuming REST/WebSocket endpoints

What’s the difference between local and remote MCP servers and their transports?

- Remote servers: typically accessed over Streamable HTTP (common in production)
- Local servers: often accessed over STDIO (useful for development or local resources)
- Regardless of location, tools integrate into agents using the same calling protocol

How widely adopted is MCP, and where can I find public servers?

- Adopted by major providers (OpenAI, Google) in APIs/SDKs since late 2024
- Public directories include:
- https://github.com/modelcontextprotocol/servers (official list)
- https://mcp.so/ (community directory with many servers)
- https://smithery.ai/ (large tool catalog, mostly MCP-compliant)
- https://mcpservers.org/ (community collection)

What resources should I read before building an MCP server?

- Protocol hub: https://modelcontextprotocol.io/
- Official specification: https://modelcontextprotocol.io/specification (covers transports, security, best practices)
- Avoid implementing from scratch; use official SDKs for your language

Which SDKs are available, and why use FastMCP 2 for Python?

- Official SDKs exist for Python, JavaScript, Java/Kotlin, and C#
- FastMCP 1: first Python SDK (https://github.com/modelcontextprotocol/python-sdk)
- FastMCP 2: major upgrade with easier deployment/composition, stronger security, better connectivity (e.g., dynamic tool rewriting), built-in testing/integration hooks
- Repo/docs: https://github.com/jlowin/fastmcp and https://gofastmcp.com/getting-started/welcome

How do I build the AccuWeather example MCP server from the chapter?

- Get an AccuWeather API key: https://developer.accuweather.com/signup
- Put the key in .env (ACCUWEATHER_API_KEY=...)
- Implement a FastMCP server exposing get_weather_conditions that resolves a location, then fetches current conditions
- Run with streamable-http on host 127.0.0.1, port 8020, path /accu-mcp-server
- Verify startup logs (Uvicorn running on http://0.0.0.0:8020)

How can I test my MCP server with MCP Inspector?

- Install and launch via: npx @modelcontextprotocol/inspector (Node.js required)
- In the UI, configure:
- Transport: Streamable HTTP
- URL: http://127.0.0.1:8020/accu-mcp-server (or http://localhost:8020/accu-mcp-server)
- Connection: Via Proxy; disable auth for local testing if needed
- Connect, list tools, select get_weather_conditions, provide a location, and run

How do I consume MCP servers in applications and agents?

- OpenAI approach: use the Responses API to discover and invoke public MCP tools (consider your authorization strategy: automatic vs. interactive approvals)
- Enterprise/intranet servers:
- Use the FastMCP Client directly (connect, list tools, call tools)
- Or use LangChain’s MultiServerMCPClient to aggregate tools from multiple MCP servers via config

How do I integrate remote MCP tools alongside local tools in an agent?

- Retrieve remote tools (e.g., via MultiServerMCPClient) and combine them with local tools in the agent’s tool list
- Use an async chat loop to support remote calls
- With LangChain: create the agent (e.g., ReAct) with tools=[local_tool, *remote_tools] and a compatible model (e.g., ChatOpenAI with use_responses_api where applicable)

pro $24.99 per month

access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
choose one free eBook per month to keep
exclusive 50% discount on all purchases
renews monthly, pause or cancel renewal anytime

lite $19.99 per month

access to all Manning books, including MEAPs!

team

5, 10 or 20 seats+ for your team - learn more

eBook

pdf, ePub, online

$47.99 $23.99

you save $24.00 (50%)

include audio $24.99 $12.49

pro $24.99 per month

access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
choose one free eBook per month to keep
exclusive 50% discount on all purchases
renews monthly, pause or cancel renewal anytime

lite $19.99 per month

access to all Manning books, including MEAPs!

team

5, 10 or 20 seats+ for your team - learn more

eBook

$47.99 $23.99

you save $24.00 (50%)

include audio $24.99 $12.49

eBook

pdf, ePub, online

$47.99 $23.99

you save $24.00 (50%)

include audio $24.99 $12.49

pro $24.99 per month

access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
choose one free eBook per month to keep
exclusive 50% discount on all purchases
renews monthly, pause or cancel renewal anytime

lite $19.99 per month

access to all Manning books, including MEAPs!

team

5, 10 or 20 seats+ for your team - learn more