table of content

Part 1 Getting started with LLMs

1 Introduction to AI agents and applications

1.1 Building LLM-based applications and agents

1.1.1 LLM-based applications: Summarization and Q&A engines

1.1.2 LLM-based chatbots

1.1.3 AI agents

1.2 Introducing LangChain

1.2.1 LangChain architecture

1.2.2 LangChain’s core object model

1.3 Typical LLM use cases

1.4 How to adapt an LLM to your needs

1.4.1 Prompt engineering

1.4.2 Retrieval-Augmented Generation

1.4.3 Fine-tuning

1.5 Which LLMs to choose

1.6 What you’ll learn from this book

1.7 LLM terminology recap

2 Executing prompts programmatically

2.1 Running prompts programmatically

2.1.1 Setting up the environment for this chapter

2.1.2 Minimal prompt execution

2.2 Running prompts with LangChain

2.3 Prompt templates

2.3.1 Implementing a prompt template with a Python function

2.3.2 Using LangChain’s PromptTemplate

2.4 Prompt types

2.4.1 Text classification

2.4.2 Sentiment analysis

2.4.3 Text summarization

2.4.4 Composing text

2.4.5 Question answering

2.4.6 Reasoning

2.5 Reasoning in detail

2.5.1 One-shot learning

2.5.2 Two-shot learning

2.5.3 Providing steps

2.5.4 Few-shot learning

2.5.5 Implementing few-shot learning with LangChain

2.5.6 Chain of Thought

2.6 Prompt structure

Part 2 Summarization

3 Summarizing text using LangChain

3.1 Summarizing a document bigger than the context window

3.1.1 Chunking the text into Document objects

3.1.2 Split

3.1.3 Map

3.1.4 Reduce

3.1.5 MapReduce combined chain

3.1.6 MapReduce execution

3.2 Summarizing across documents

3.2.1 Creating a list of Document objects

3.2.2 Wikipedia content

3.2.3 File-based content

3.2.4 Creating the Document list

3.2.5 Progressively refining the final summary

3.3 Summarization flowchart

4 Building a research summarization engine

4.1 Overview of a research summarization engine

4.2 Setting up the project

4.3 Implementing the core functionality

4.3.1 Implementing web searching

4.3.2 Implementing web scraping

4.3.3 Instantiating the LLM client

4.3.4 JSON to Python object converter

4.4 Enhancing the architecture with query rewriting

4.5 Prompt engineering

4.5.1 Crafting web search prompts

4.5.2 Crafting summarization prompts

4.5.3 Research report prompt

4.6 Initial implementation

4.6.1 Importing functions and prompt templates

4.6.2 Setting constants and input variables

4.6.3 Instantiating the LLM client

4.6.4 Generating the web searches and collecting the results

4.6.5 Scraping the web results

4.6.6 Summarizing the web results

4.6.7 Generating the research report

4.7 Reimplementing the research summary engine in LCEL

4.7.1 Assistant Instructions chain

4.7.2 Web Searches chain

4.7.3 Search and Summarization chain

4.7.4 Web Research chain

5 Agentic workflows with LangGraph

5.1 Understanding agentic workflows and agents

5.1.1 Workflows

5.1.2 Agents

5.1.3 When to use agent-based architectures

5.1.4 Agent development frameworks

5.2 LangGraph basics

5.3 Moving from LangChain chains to LangGraph

5.4 LangGraph core components

5.4.1 StateGraph structure

5.4.2 State management and typing

5.4.3 Node functions and edge definitions

5.4.4 Entry points and end conditions

5.5 Turning the web research assistant into an AI agent

5.5.1 Original LangChain implementation overview

5.5.2 Identifying components for conversion

5.5.3 Step-by-step transformation process

5.5.4 Code comparison and benefits realized

Part 3 Q&A chatbots

6 RAG fundamentals with ChromaDB

6.1 Semantic search

6.1.1 A basic Q&A chatbot over a single document

6.1.2 A more complex Q&A chatbot over a knowledge base

6.1.3 The RAG design pattern

6.2 Vector stores

6.2.1 What’s a vector store?

6.2.2 How do vector stores work?

6.2.3 Vector libraries vs. vector databases

6.2.4 Most popular vector stores

6.2.5 Storing text and performing a semantic search using Chroma

6.3 Implementing RAG from scratch

6.3.1 Retrieving content from the vector database

6.3.2 Invoking the LLM

6.3.3 Building the chatbot

6.3.4 Recap of RAG terminology

7 Q&A chatbots with LangChain and LangSmith

7.1 LangChain object model for Q&A chatbots

7.1.1 Content ingestion (indexing) stage

7.1.2 Q&A (retrieval and generation) stage

7.2 Vector store content ingestion

7.2.1 Splitting and storing the documents

7.2.2 Removing duplication

7.2.3 Ingesting multiple documents from a folder

7.3 Q&A across stored documents

7.3.1 Querying the vector store directly

7.3.2 Asking a question through a LangChain chain

7.3.3 Completing the RAG chain setup

7.3.4 Follow-up question

7.4 Chatbot memory of message history

7.4.1 Amending the prompt

7.4.2 Updating the chat message history

7.4.3 Feeding the chat history to the RAG chain

7.4.4 Putting everything together

7.5 Tracing execution with LangSmith

Part 4 Advanced RAG

8 Advanced indexing

8.1 Improving RAG accuracy

8.1.1 Content ingestion stage

8.1.2 Question-answering stage

8.2 Advanced document indexing

8.3 Splitting strategy

8.3.1 Splitting strategies

8.3.2 Factors to consider

8.3.3 Choosing the right strategy

8.3.4 Splitting by HTML header

8.4 Embedding strategy

8.4.1 Embedding child chunks with ParentDocumentRetriever

8.4.2 Embedding child chunks with MultiVectorRetriever

8.4.3 Embedding document summaries

8.4.4 Embedding hypothetical questions

8.5 Granular chunk expansion

8.6 Semi-structured content

8.7 Multimodal RAG

9 Question transformations

9.1 Rewrite-Retrieve-Read

9.1.1 Retrieving content using the original user question

9.1.2 Setting up the query rewriter chain

9.1.3 Retrieving content with the rewritten query

9.1.4 Combining everything into a single RAG chain

9.2 Generating multiple queries

9.2.1 Setting up the chain for generating multiple queries

9.2.2 Setting up a custom multi-query retriever

9.2.3 Using a standard MultiQueryRetriever instance

9.3 Step-back question

9.3.1 Setting up the chain to generate a step-back question

9.3.2 Incorporating step-back question generation into the RAG chain

9.4 Hypothetical Document Embeddings (HyDE)

9.4.1 Generating a hypothetical document for the user question

9.4.2 Integrating the HyDE chain into the RAG chain

9.5 Single-step and multi-step decomposition

10 Query generation, routing, and retrieval postprocessing

10.1 Content database query generation

10.2 Self-querying (metadata query enrichment)

10.2.1 Ingestion: Metadata enrichment

10.2.2 Q&A on a metadata-enriched collection

10.3 Generating a structured SQL query

10.3.1 Installing SQLite

10.3.2 Setting up and connecting to the database

10.3.3 Generating SQL queries from natural language

10.3.4 Executing the SQL query

10.4 Generating a semantic SQL query

10.4.1 Standard SQL query

10.4.2 Semantic SQL query

10.4.3 Creating the embeddings

10.4.4 Performing a semantic SQL search

10.4.5 Automating semantic SQL search

10.4.6 Benefits of a semantic SQL search

10.5 Generating queries for a graph database

10.6 Chain routing

10.6.1 Setting up data retrievers

10.6.2 Setting up the query router

10.6.3 Integrating the chain router into a full RAG chain

10.7 Retrieval postprocessing

10.7.1 Similarity postprocessors

10.7.2 Keyword postprocessors

10.7.3 Time weighting

10.7.4 RAG fusion (Reciprocal Rank Fusion)

Part 5 AI agents

11 Building tool-based agents with LangGraph

11.1 Starting simple: Building a single-tool travel info agent

11.1.1 Project setup

11.1.2 Loading environment variables

11.1.3 Preparing the travel information vector store

11.2 Enabling agents to call tools

11.2.1 From function calling to tool calling

11.2.2 How tool calling works with LLMs

11.2.3 Registering tools with the LLM

11.2.4 Agent state: Tracking the conversation

11.2.5 Executing tool calls

11.2.6 The LLM node: Coordinating reasoning and action

11.3 Assembling the agent graph

11.4 Understanding the agent graph structure

11.5 Running the agent chatbot: The Read-Eval-Print Loop

11.6 Executing a request

11.6.1 Step-by-step debugging

11.7 Expanding your agent: Adding a weather forecast tool

11.7.1 Implementing a mock weather service

11.7.2 Creating the weather forecast tool

11.7.3 Updating the agent for multi-tool support

11.8 Executing the multi-tool agent

11.8.1 Running the multi-tool agent (initial behavior)

11.8.2 Improving LLM tool usage with system guidance

11.9 Using prebuilt components for rapid development

11.9.1 Refactoring to use the LangGraph ReAct agent

11.9.2 Running the prebuilt agent

11.9.3 Observing and debugging with LangSmith

11.9.4 Enabling LangSmith tracing

12 Multi-agent systems

12.1 Building an accommodation booking agent

12.1.1 Hotel booking tool

12.1.2 B&B booking tool

12.1.3 ReAct accommodation booking agent

12.2 Building a router-based travel assistant

12.2.1 Designing the router agent

12.2.2 Routing logic

12.2.3 Building the multi-agent graph

12.2.4 Trying out the router agent

12.3 Handling multi-agent requests with a Supervisor component

12.3.1 The Supervisor pattern: An agent of agents

12.3.2 From “one-way” to “return ticket” interactions

12.3.3 Trying out the Supervisor agent

13 Building and consuming MCP servers

13.1 Introduction to MCP servers

13.1.1 The problem: Context integration at scale

13.1.2 The solution: The Model Context Protocol

13.1.3 The MCP ecosystem

13.2 How to build MCP servers

13.2.1 Essential resources for MCP server development

13.2.2 Official language-specific MCP SDKs

13.2.3 Consuming MCP servers in LLM applications and agents

13.3 Building a weather MCP server

13.3.1 Implementing the MCP server

13.3.2 Trying out the MCP server with MCP Inspector

13.3.3 Consuming the MCP server from a test MCP host

13.4 Integrating the Weather MCP tool into an agent

13.4.1 Preparing the travel agent for live weather data

13.4.2 Integrating the AccuWeather MCP tool

13.4.3 Updating the agent chat loop

13.4.4 Combining local and remote tools

13.4.5 Testing and verification

13.4.6 Using the agent for complex queries

14 Productionizing AI agents: Memory, guardrails, and beyond

14.1 Memory

14.1.1 Types of memory

14.1.2 Why short-term memory is needed

14.1.3 Checkpoints in LangGraph

14.1.4 Adding short-term memory to our travel assistant

14.1.5 Executing the checkpointer-enabled assistant

14.1.6 Rewinding the state to a past checkpoint

14.2 Guardrails

14.2.1 Implementing guardrails to reject nontravel-related questions

14.2.2 Implementing more restrictive guardrails at the agent level

14.3 Beyond this chapter

14.3.1 Long-term user and application memory

14.3.2 Human-in-the-loop

14.3.3 Post-model guardrails

14.3.4 Evaluation of AI agents and applications

14.3.5 Deployment on the LangGraph platform and Open Agent Platform

Appendices

Appendix A: Trying out LangChain

A.1 Trying out LangChain in a Jupyter Notebook environment

A.1.1 Sentence completion example

A.1.2 Prompt engineering examples

A.1.3 Creating chains and executing them with LCEL

Appendix B: Setting up a Jupyter Notebook environment

B.1 Installing the Python interpreter or a Python distribution

B.2 Creating a virtual environment with venv and upgrading pip

B.3 Setting up a Jupyter Notebook

Appendix C: Choosing an LLM

C.1 Popular large language models

C.1.1 OpenAI GPT series

C.1.2 Gemini

C.1.3 Gemma

C.1.4 Claude

C.1.5 Cohere

C.1.6 Llama

C.1.7 Falcon

C.1.8 Mistral

C.1.9 Qwen

C.1.10 Grok

C.1.11 Phi

C.1.12 DeepSeek

C.2 How to choose a model

C.2.1 Model purpose

C.2.2 Proprietary vs. open source

C.2.3 Model size (number of parameters)

C.2.4 Context window size

C.2.5 Multilingual support

C.2.6 Accuracy vs. speed

C.2.7 Cost and hardware requirements

C.2.8 Task suitability (standard benchmarks)

C.2.9 Safety and Bias

C.2.10 A practical example

C.3 A word of caution

Appendix D: Installing SQLite on Windows

D.1 Installing SQLite

Appendix E: Open source LLMs

E.1 Benefits of open source LLMs

E.1.1 Transparency

E.1.2 Privacy

E.1.3 Community driven

E.1.4 Cost savings

E.2 Popular open source LLMs

E.3 Considerations on running open source LLMs locally

E.3.1 Limitations of consumer hardware

E.3.2 Quantization

E.3.3 OpenAI REST API compatibility

E.4 Local inference engines

E.4.1 llama.cpp

E.4.2 Ollama

E.4.3 vLLM

E.4.4 llamafile

E.4.5 LM Studio

E.4.6 LocalAI

E.4.7 GPT4All

E.4.8 Comparing local inference engines

E.4.9 Choosing a local inference engine

E.5 Inference via the Hugging Face Transformers library

E.5.1 Hugging Face Transformers library

E.5.2 LangChain’s Hugging Face pipeline

E.6 Building a local summarization engine

E.6.1 Choosing the inference engine

E.6.2 Starting up the OpenAI-compatible server

E.6.3 Modifying the original solution

E.6.4 Running the summarization engine through the local LLM

E.6.5 Comparison between OpenAI and local LLM

Overview

Appendix B. Setting up a Jupyter Notebook environment

This appendix provides a practical, high-level guide to getting a Jupyter Notebook environment ready for experimenting with AI prompts and examples. It begins by framing notebooks as an interactive space for writing and running code with immediate feedback, and recommends using Python 3.11 or newer for the smoothest experience. It then outlines three ways to install Python: a lightweight standalone interpreter, the full Anaconda distribution with extensive data science tools and an easy GUI, and the slimmer Miniconda option focused on environment management. The remaining instructions assume a standard Python installation, while conda users can adapt the steps as needed.

Next, the appendix walks through creating an isolated project setup using venv to prevent package conflicts across projects. You create a project folder, initialize a virtual environment, activate it so the shell reflects the environment name, and upgrade pip to ensure reliable package installation. This isolation keeps dependencies clean and reproducible, making it easier to manage versions and avoid interference with other work on the same machine.

With the environment active, you install Jupyter along with the required libraries (including those used for LangChain examples) from a requirements file, optionally verifying the installation with a package list. You then launch Jupyter from the terminal, which starts a local server and opens the notebook interface in your browser. From there, you create a new notebook, rename it to prompt_examples.ipynb, and begin entering code into cells—remembering that Ctrl+Enter executes the current cell—so you’re ready to run prompts and iterate on examples immediately.

Figure B.1 Creating a new Jupyter notebook: create the notebook with the menu File > New > Notebook and then rename the file to langchain_examples.ipynb

FAQ

What Python version is recommended for these Jupyter notebooks?

The examples are version-agnostic, but Python 3.11 or newer is recommended for the smoothest experience.

Should I install standalone Python, Anaconda, or Miniconda?

- Standalone Python: Lightweight install from python.org; ideal if you just want Python without extra tools.
- Anaconda: Full data science stack (Python + many libraries + Navigator); great if you have disk space and want an all-in-one setup.
- Miniconda: Lightweight conda with just the essentials; add only what you need via environments.

How do I create and activate a virtual environment with venv?

1) In a terminal, create/navigate to your project folder (for example, ch01).
2) Create the environment: python -m venv env_ch01
3) Activate it (Windows): .\env_ch01\Scripts\activate
When activated, your prompt will show (env_ch01) at the beginning.

Why should I upgrade pip and how do I do it?

Upgrading pip helps avoid installation issues with newer packages. Run: python -m pip install --upgrade pip inside your activated environment.

Where can I find official guidance on pip and virtual environments?

See the Python Packaging guide: https://packaging.python.org/en/latest/guides/installing-using-pip-and-virtual-environments/.

How do I install Jupyter and the book’s required libraries?

With your environment active, run: pip install -r requirements.txt. Use the requirements file from the book’s repository: https://github.com/roberto-inf/building-llm-applications.

How can I verify that packages installed correctly?

Run pip list in the activated environment to see installed packages and versions.

How do I start Jupyter Notebook and access it in the browser?

Start it with jupyter notebook. The terminal will show a URL like http://localhost:8888/tree?token=.... A browser window should open automatically; if not, copy that URL into your browser.

I see a token in the terminal. What is it and how do I use it?

The token is an access key for your local Jupyter session. Either click the full URL printed in the terminal (which includes the token) or paste that URL into your browser. You can stop the server anytime with Ctrl+C in the terminal.

How do I create and rename my notebook file, and run code cells?

In Jupyter: File > New > Notebook, then File > Rename… and name it prompt_examples.ipynb. To execute a cell, press Ctrl+Enter.

pro $24.99 per month

access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
choose one free eBook per month to keep
exclusive 50% discount on all purchases
renews monthly, pause or cancel renewal anytime

lite $19.99 per month

access to all Manning books, including MEAPs!

team

5, 10 or 20 seats+ for your team - learn more

eBook

pdf, ePub, online

$47.99 $23.99

you save $24.00 (50%)

include audio $24.99 $12.49

pro $24.99 per month

access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
choose one free eBook per month to keep
exclusive 50% discount on all purchases
renews monthly, pause or cancel renewal anytime

lite $19.99 per month

access to all Manning books, including MEAPs!

team

5, 10 or 20 seats+ for your team - learn more

eBook

$47.99 $23.99

you save $24.00 (50%)

include audio $24.99 $12.49

eBook

pdf, ePub, online

$47.99 $23.99

you save $24.00 (50%)

include audio $24.99 $12.49

pro $24.99 per month

access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
choose one free eBook per month to keep
exclusive 50% discount on all purchases
renews monthly, pause or cancel renewal anytime

lite $19.99 per month

access to all Manning books, including MEAPs!

team

5, 10 or 20 seats+ for your team - learn more