table of content

Part 1 Reliable outputs

1 AI reliability: Building LLMs for the real world

1.1 From benchmarks to the real world

1.2 The tangible impact of LLMs in the real world

1.2.1 Legal industry

1.2.2 Customer service

1.2.3 Programming and development

1.2.4 Enterprise AI

1.3 Understanding hallucinations and reliable AI

1.3.1 When AI hallucinates convincingly

1.3.2 What is a hallucination?

1.3.3 What is reliable AI?

1.4 The AI reliability framework

1.4.1 Layer 1: Reliable outputs

1.4.2 Layer 2: Reliable agents

1.4.3 Layer 3: Reliable operations

1.5 The reliability toolbox

1.6 Why reliable AI systems matter now

1.7 Requirements for following along

2 Generating trustworthy responses with prompt engineering

2.1 Choosing the right model

2.1.1 Reasoning vs. nonreasoning models

2.1.2 The cost-capability spectrum

2.1.3 Model selection as a reliability decision

2.2 Tailoring LLM settings for maximum reliability

2.2.1 Optimizing temperature for predictable outputs

2.2.2 Top-P Sampling technique

2.3 Limiting output and turns to reduce hallucination risk

2.4 Applying frequency and presence penalties for balanced content

2.5 Minimizing intrinsic randomness for stable performance

2.6 Foundations of prompt engineering for reliable LLMs

2.6.1 Designing components of a prompt for reliability

2.6.2 Crafting basic prompts with a focus on dependability

2.7 Prompt engineering techniques for preventing hallucinations

2.7.1 Zero-shot prompting

2.7.2 Few-shot prompting for contextual stability

2.7.3 Chain-of-thought prompting for transparent reasoning

2.7.4 Enhanced CoT prompt

2.7.5 Automatic CoT for scalable reasoning

2.7.6 Self-consistency for cross-verification

2.7.7 Tree-of-thought prompting for structured decision-making

2.8 Project: Creating a reliable weather assistant with OpenAI’s function calls

2.8.1 Building a weather assistant

2.8.2 Getting started and defining the weather function

2.8.3 Integrating the chatbot

3 Grounding outputs with RAG

3.1 What is RAG?

3.2 RAG system architecture

3.2.1 The role of the retriever

3.2.2 The factual generators

3.2.3 RAG system flow

3.2.4 Example: E-commerce shopping assistant

3.3 Reducing hallucinations with RAG

3.3.1 Grounding the language model

3.3.2 Grounding an e-commerce product query

3.3.3 Advantages of RAG in reducing hallucinations

3.3.4 Critical applications

3.3.5 Enhancing transparency

3.4 Data preparation and reliable indexing for RAG systems

3.4.1 Introducing LangChain

3.4.2 Structuring product catalogs

3.4.3 Creating the searchable vector index

3.4.4 Implementing an index

3.5 Building an effective RAG system

3.5.1 Building an e-commerce chatbot powered by RAG

3.5.2 Processing user queries

3.5.3 Crafting an accurate response using an LLM

3.6 Evaluating and optimizing RAG systems

3.6.1 The role of evaluator LLMs in assessing hallucinations

3.6.2 Crafting evaluation datasets

3.6.3 Practical metrics for RAG evaluation

3.6.4 Implementing RAGAS evaluation

3.6.5 Reducing hallucinations by filtering retrievals and using metadata

3.6.6 Metadata-based filtering in a telecom chatbot

3.6.7 Embedding chunk sizes and model choice

3.7 Advanced techniques for improving RAG

3.7.1 Fine-tuning embeddings for in-domain relevance

3.7.2 Incorporating metadata to reestablish context

3.7.3 Implementing hybrid retrieval

3.7.4 Enhancing queries via rephrasing and augmentation

3.7.5 Implementing query routing

3.7.6 GraphRAG: Knowledge graph-enhanced retrieval

3.7.7 When to use GraphRAG vs. traditional RAG

3.7.8 Agentic RAG: Agent-controlled retrieval

3.8 Building an RAG system for an e-commerce chatbot

3.8.1 Step 1: Install the required libraries

3.8.2 Step 2: Import libraries

3.8.3 Step 3: Load the documents

3.8.4 Step 4: Document transformers

3.8.5 Step 5: Embed text

3.8.6 Step 6: Choose a vector store

3.8.7 Step 7: Prepare the LLM model

3.8.8 Step 8: Add a retriever

3.8.9 Step 9: Create a retrieval QA chain

3.8.10 Step 10: Test the chatbot

3.8.11 Step 11: Evaluate and prevent hallucinations with LangChain

3.8.12 Challenges and adaptations of the chatbot

4 Embeddings and vector search

4.1 What are embeddings?

4.1.1 A mental model: Embeddings as coordinates in meaning space

4.1.2 Why the model matters

4.2 Embedding models in production

4.2.1 Commercial models: Powerful but opaque

4.2.2 Open source models: Flexible and transparent

4.2.3 Domain-specific models: Purpose-built precision

4.2.4 How to choose the right model

4.2.5 Embeddings across industries

4.2.6 Case study: Learning from Airbnb’s embedding journey

4.3 Beyond simple vectors: Hybrid and multistage retrieval

4.3.1 The limitations of pure vector search

4.3.2 Hybrid retrieval: Combining dense and sparse approaches

4.3.3 Multistage retrieval: Building precision through layers

4.3.4 Building a hybrid retriever: Hands-on implementation for an employee chatbot

4.4 Essential retrieval optimizations with embeddings and RAG

4.4.1 Metadata filtering: Beyond pure semantic search

4.4.2 Chunking strategies: Finding the right granularity

4.5 Exploring vector storage and databases

4.5.1 HNSW: The algorithm powering modern vector search

4.5.2 FAISS: Industry-standard vector indexing

4.5.3 Vector databases: Complete vector storage solutions

4.5.4 Vector database project: Building a health-care policy assistant with Pinecone

4.6 Practical challenges: Drift and compression

4.6.1 Embedding drift explained

4.6.2 Diving deeper into vector compression

5 Fine-tuning LLMs for improved performance

5.1 Choosing the right approach: Prompting, RAG, or fine-tuning

5.1.1 When to use each approach: A practical decision framework

5.1.2 Hybrid approach: Combining RAG and fine-tuning

5.2 Case studies: Fine-tuning in the real world

5.2.1 Medicine: Med-PaLM 2

5.2.2 Finance: BloombergGPT and FinGPT

5.2.3 Code generation: Codex

5.3 Understanding the fine-tuning process

5.3.1 Data preparation

5.3.2 Fine-tuning closed-source models (OpenAI)

5.3.3 Understanding key fine-tuning approaches

5.3.4 Building a customer-support assistant

5.4 Knowledge distillation for LLMs

5.4.1 How knowledge distillation works in LLMs

5.4.2 Best practices

Part 2 Reliable agents

6 Creating effective AI agents

6.1 Why do we need agents?

6.1.1 Static knowledge

6.1.2 Inability to act

6.1.3 No workflow management

6.1.4 How AI agents solve these challenges

6.1.5 Broader applications of AI agents

6.2 Core components of reliable agents

6.2.1 Memory: Context that sticks

6.2.2 Balancing memory systems with LangChain

6.2.3 Active tool use vs. passive retrieval

6.2.4 Strategies for reducing hallucinations with tools

6.2.5 Decision-making: Making AI agents intelligent

6.3 Agentic RAG for reliable agents

6.3.1 Project: Building an agentic RAG system with LangChain

6.3.2 Setting up dependencies

6.3.3 Building the knowledge base

6.3.4 Creating interactive tools

6.3.5 Configuring the agent’s behavior

6.3.6 Implementing transparent decision-making

6.3.7 Managing conversation context

6.3.8 Making the system work in production

6.4 Advanced techniques

6.4.1 Cross-referencing

6.4.2 Guardrails for security and preventing hallucinations

6.4.3 Reinforcement Learning from AI Feedback

6.4.4 Transparency in multistep reasoning

6.4.5 Case studies for agents in production

7 Tool integration and MCP

7.1 What is MCP?

7.2 The hidden complexity: The N×M problem

7.3 The solution: MCP

7.4 Building your first real MCP tool with a CSV-powered product catalog

7.4.1 Loading your product catalog

7.4.2 Setting up the MCP server

7.5 Running and testing your server

7.5.1 Why this matters

7.5.2 Next steps

7.6 Teaching your AI to use MCP tools

7.6.1 How models learn what tools they can use

7.6.2 A complete example: Model > MCP > answer

7.6.3 What you didn’t have to write

7.6.4 Tool design becomes interface design

7.7 Adding a second tool: Inventory status

7.7.1 How models chain tools

7.7.2 Tool descriptions guide model behavior

7.8 Handling failures gracefully

7.8.1 Catching and communicating errors

7.8.2 Why structured responses matter

7.8.3 Handling empty results vs. errors

7.8.4 Timeout and rate-limiting

7.9 Beyond tools: Production agent harnesses and OpenClaw

7.9.1 Why harnesses matter

7.9.2 Designing agent-legible environments

7.9.3 The security gap

8 Multi-agent systems

8.1 Multi-agent systems

8.1.1 Why multi-agent architecture matters

8.1.2 Introducing LangGraph: The multi-agent framework

8.1.3 Building intelligent agent coordination

8.1.4 Real-world example: ShopBot multi-agent system

8.1.5 Running your multi-agent system

8.1.6 Adding vision: Image-based product search with VisionAgent

8.1.7 What you built and why it matters

8.2 Testing multi-agent workflows

8.2.1 Category 1: Blended queries and multimodal interactions

8.2.2 Category 2: Ambiguous intent and robust routing decisions

8.2.3 Category 3: Agent failure and resilience verification

8.2.4 Category 4: State flow and information preservation across the workflow

8.2.5 Regression: Moving forward without breaking what worked

8.3 Alternative framework: CrewAI

8.3.1 LangGraph vs. CrewAI: Different philosophies

8.3.2 ShopBot with CrewAI

8.3.3 When to use which multi-agent framework

8.3.4 The framework landscape

Part 3 Reliable operations

9 Evaluation and performance for LLMs and agents

9.1 Identifying and measuring hallucinations

9.1.1 Four steps to identify and measure hallucinations

9.1.2 FActScore: fine-grained factual evaluation

9.1.3 ROUGE for summarization evaluation

9.1.4 LLM-as-a-judge: A holistic approach

9.1.5 Red teaming and stress testing

9.1.6 Using monitoring frameworks to detect hallucinations

9.2 Essential architectural patterns for performance

9.2.1 Token streaming: Presenting answers incrementally or at the same time

9.2.2 Handling surges with batching: System-level vs. OpenAI’s Batch API

9.2.3 Caching for efficiency

9.2.4 Multimodel fallback: Matching each query to the right model

9.2.5 Project: Building an e-commerce LLM service with batching, caching, and model fallback

9.2.6 Further improvements

9.3 Evaluating agent performance

9.3.1 Core metrics for evaluating LLMs

9.3.2 Evaluating agents: Beyond traditional LLM metrics

10 Deploying and monitoring

10.1 Introducing large language model operations

10.2 Serving LLMs: Hosted APIs vs. open source models

10.2.1 Using hosted APIs

10.2.2 The open source alternative

10.2.3 The hybrid solution: Best of both worlds

10.3 Building LLM-native monitoring systems

10.3.1 What really matters: The four questions

10.3.2 Logging what matters

10.3.3 Setting up alerts that help

10.3.4 Catching cost explosions before they hurt

10.3.5 Building dashboards that drive action

10.3.6 Output-quality monitoring

10.4 User experience and feedback monitoring

10.4.1 Explicit feedback collection

10.4.2 Implicit feedback signals

10.4.3 Building actionable feedback loops

10.5 Ensuring high-quality outputs in production

10.5.1 The three-pillar quality framework

10.5.2 Prompt engineering for consistent quality

10.5.3 Continuous quality monitoring with automated testing

10.6 Observability in practice: Introducing Langfuse

11 Bias, privacy, and responsible AI

11.1 The responsible AI imperative

11.1.1 Regulatory pressure is accelerating

11.1.2 User expectations have shifted

11.1.3 Business risks have multiplied

11.1.4 Real examples of AI bias in production

11.1.5 The four failure modes

11.1.6 The responsible AI defense system

11.2 Data layer: Where bias begins

11.2.1 The fine-tuning bias trap

11.2.2 Detecting bias in chat logs

11.2.3 The name experiment

11.2.4 Three proven bias-mitigation strategies

11.3 Model layer: Where bias evolves

11.3.1 Why this matters for open source models

11.3.2 Example: Adding fairness to a LoRA fine-tuning loop

11.3.3 Anthropic’s constitutional AI: LLM-as-a-judge at training scale

11.4 Safety layer: Your last line of defense

11.4.1 Multilayered safety architecture

11.4.2 Layer 3: Enhanced safety with commercial APIs

11.5 Privacy layer: Protecting personal data

11.5.1 Why LLM privacy failures are uniquely dangerous

11.5.2 Building sensitive data detection

11.5.3 Health-care privacy protection

11.5.4 European data protection

11.6 Real-world project: SafeMedAssist

11.6.1 Why build a medical AI assistant?

11.6.2 Professional testing with LangTest

11.6.3 Production-deployment considerations

11.6.4 The business case for responsible AI

11.6.5 Completing the reliability framework

11.6.6 The six principles of reliable AI

11.6.7 Final thoughts

references

bibliography

Overview

7 Tool Integration and MCP

This chapter explains how the Model Context Protocol helps AI systems connect to real-world tools and data sources without creating a fragile web of custom integrations. Instead of writing separate glue code for every combination of AI application and external API, MCP standardizes the interface between models and tools through a client-server pattern. External systems expose capabilities as MCP servers, while AI applications connect through MCP clients, reducing integration complexity from an exponential problem to a more modular and predictable workflow.

The chapter then walks through building a practical MCP-powered product catalog tool. Using a CSV file as a simple product database, it shows how a Python server can expose a search_products function that filters products by query, price, availability, and result limit. The tool returns structured data that language models can interpret and use in customer-facing responses. A second inventory-checking tool demonstrates how models can chain multiple tools together, such as searching for products first and then verifying whether a specific item is in stock.

The chapter also emphasizes that reliable tool integration requires thoughtful design beyond simply exposing functions. Clear tool names, descriptions, schemas, structured outputs, and consistent error responses help models decide when to call tools and how to recover from failures. It further introduces the idea of production agent harnesses, which manage state, approvals, routing, tracing, memory, and sandboxed execution around the model. The main takeaway is that MCP provides the standardized “hands” for AI systems, while secure harnesses and well-designed environments make those tools safe, reliable, and useful in production.

Without MCP (left), each application must individually integrate with each external API, creating N×M integrations. On the right, the MCP architecture shows the scalability benefits of standardized client-server interactions.

How AI discovers and uses MCP tools: once tools are registered on the server, the AI can automatically discover, reason about, and invoke them.

Comparison of poor vs well-designed MCP tool schemas. The left example shows a vague tool definition that confuses AI models, while the right example demonstrates clear naming, descriptions, and self-documenting parameters that enable intelligent tool usage.

The harness owns the agent loop, memory, and LLM communication. The sandbox executes code in isolation with no direct model access. Tasks flow right; results flow left. If the sandbox crashes, the harness recovers.

Summary

The N×M integration problem creates exponential complexity when connecting AI applications to external systems. Five apps × ten APIs = fifty custom integrations.
Model Context Protocol (MCP) provides a standardized way for language models to interact with external tools and APIs, eliminating custom glue code through structured schemas. MCP reduces exponential complexity to linear growth (5 apps + 10 MCP servers = 15 connectors).
MCP tools are self-describing: LLMs can discover, understand, and call them automatically by reading their schemas from the MCP server's /tools/list endpoint.
To implement MCP, you define Python functions that perform useful tasks, decorate them as tools using FastMCP, and run them on a server that exposes their interfaces in a machine-readable format.
Tool design is interface design: clear names, descriptions, and parameter schemas help models understand when and how to use each tool effectively.
Multiple MCP tools can work together through tool chaining where the model automatically determines which tools to call and in what order based on user intent.
Production-grade MCP tools require structured error handling that distinguishes between failures (database down) and empty results (no matches found).
Consistent response structures with success flags, error codes, and human-readable messages help models provide helpful responses even when things go wrong.
Timeouts and input validation protect your tools from hanging indefinitely or processing invalid requests.
Production agent systems separate the harness (control plane: agent loop, approvals, tracing, state) from the sandbox (compute plane: file I/O, shell, code execution). This pattern keeps credentials out of environments where model-generated code runs.
Agent-legible environments use structured documentation (like AGENTS.md files) as a table of contents rather than a monolithic instruction manual, enabling progressive disclosure where agents start with a small map and follow pointers to depth.
Agent adoption is outpacing agent security: most organizations use agents but few have full security approval. Design your MCP tools with the assumption that prompt injection and data exfiltration will be attempted.

FAQ

What is the Model Context Protocol (MCP)?

MCP is a standard protocol for connecting AI systems to external tools, APIs, databases, and services through clean, standardized interfaces. Instead of writing custom integration code for every AI application and every external system, MCP lets tools expose a common server interface that AI applications can discover and call.

What problem does MCP solve in AI systems?

MCP solves the N×M integration problem. Without MCP, every AI application must integrate separately with every external API. For example, 5 AI apps connecting to 10 APIs require 50 custom integrations. With MCP, each app connects through a standard MCP client and each API exposes an MCP server, reducing the work to a more linear and reusable integration model.

How is MCP like “USB for AI”?

MCP is compared to USB because it provides a standard way to plug tools into AI systems. Once a tool exposes an MCP-compatible interface, an AI model or agent framework can discover and use it without requiring custom glue code for every new integration.

What are MCP servers and MCP clients?

An MCP server exposes tools or external systems through the MCP standard, such as a product catalog, payment processor, email service, or inventory system. An MCP client is used by an AI application to connect to those servers using the same common protocol. Together, they replace many one-off integrations with standardized client-server interactions.

Does MCP automatically handle security for tools?

No. MCP standardizes the interface, schema validation, and error formatting, but developers are still responsible for authentication, authorization, endpoint security, approval policies, and protecting sensitive operations. In production, tools that modify data, send messages, or process payments should usually require human approval or custom approval logic.

How do you build a simple MCP product search tool?

The chapter demonstrates building a CSV-powered product catalog using Python, FastMCP, and pandas. A products.csv file stores product data such as id, title, price, and available. A FastMCP server exposes a search_products tool that accepts a query, optional maximum price, and result limit, then returns matching in-stock products as JSON-like structured data.

Why does the product search tool return structured data instead of plain text?

Structured data is easier for language models to reason over. Returning a list of dictionaries or a consistent object with fields such as success, error, message, and results lets the model check whether the tool worked, explain results to the user, retry intelligently, or suggest alternatives.

How do AI models discover MCP tools automatically?

MCP servers provide a machine-readable tool list through a special endpoint such as /tools/list. When an AI runtime is given access to an MCP server, it can query that endpoint, load the tool names, descriptions, input parameters, and schemas into the conversation context, and then decide when and how to call those tools based on user intent.

How can multiple MCP tools work together?

Models can chain MCP tools automatically when a user request requires multiple steps. For example, if a customer asks, “Do you have any waterproof jackets? Is the cheapest one in stock?”, the model can first call search_products to find matching jackets, then call check_inventory with the selected product ID, and finally produce a grounded answer.

Why are agent harnesses important beyond individual MCP tools?

An MCP tool is only one capability. An agent harness provides the surrounding infrastructure that turns a model into a working agent: session state, memory, tool routing, approvals, tracing, recovery, and communication with external systems. The chapter emphasizes separating the harness, which controls state and credentials, from sandboxed execution environments, where generated code or risky operations can run in isolation.

pro $24.99 per month

access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
choose one free eBook per month to keep
exclusive 50% discount on all purchases
renews monthly, pause or cancel renewal anytime

lite $19.99 per month

access to all Manning books, including MEAPs!

team

5, 10 or 20 seats+ for your team - learn more

eBook

pdf, ePub, online

$47.99 $23.99

you save $24.00 (50%)

include audio $24.99 $12.49

pro $24.99 per month

access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
choose one free eBook per month to keep
exclusive 50% discount on all purchases
renews monthly, pause or cancel renewal anytime

lite $19.99 per month

access to all Manning books, including MEAPs!

team

5, 10 or 20 seats+ for your team - learn more

eBook

$47.99 $23.99

you save $24.00 (50%)

include audio $24.99 $12.49

eBook

pdf, ePub, online

$47.99 $23.99

you save $24.00 (50%)

include audio $24.99 $12.49

pro $24.99 per month

access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
choose one free eBook per month to keep
exclusive 50% discount on all purchases
renews monthly, pause or cancel renewal anytime

lite $19.99 per month

access to all Manning books, including MEAPs!

team

5, 10 or 20 seats+ for your team - learn more