6 Creating Effective AI Agents

AI agents extend standalone LLMs from passive text generators into systems that can retrieve current information, take actions, and coordinate multi-step workflows. The chapter explains that LLMs are limited by static knowledge, inability to interact with external systems, and lack of workflow management. Agents address these gaps by connecting to APIs, databases, and tools; executing tasks such as scheduling meetings or booking travel; and adapting their actions based on context, user preferences, and intermediate results.

Reliable agents depend on three core capabilities: memory, tool integration, and decision-making. Memory allows agents to retain session context and, when appropriate, long-term user preferences, though it must be balanced against privacy, scalability, and context-window limits. Tool integration grounds responses in real-world data and reduces hallucinations by letting agents verify information through trusted sources. Decision-making frameworks, including rule-based logic, AI-driven reasoning, ReAct, and Tree-of-Thought approaches, help agents select tools, sequence actions, observe results, and adjust their behavior as tasks evolve.

The chapter also shows how agentic RAG combines retrieval with real-time actions, using a travel assistant example that can answer stored-knowledge questions while also checking live weather through tools. For production use, it emphasizes focused tools, clear tool descriptions, error handling, validation, fallback logic, cross-referencing, guardrails, transparency, and feedback loops to improve trust and reliability. Real-world case studies in healthcare, finance, document processing, and retail demonstrate that agents are moving from prototypes to deployed systems, where they reduce manual work, improve productivity, and support complex business workflows at scale.

Core differences between a Standalone LLM and AI Agent

Travel Agent LLM Flow starts from query and finishes booking the flight or suggests alternatives

The 3 core components of building a reliable agent

The two types of memory in agents: Short-term and Persistent Memory

The ReAct Framework for reasoning

An Agentic RAG system consisting of a Weather Tool (API) and Vector Store (RAG)

Example: Perplexity displays the steps that the agent used to answer the question as well as citations

• Standalone LLMs like GPT-5 are powerful but face key limitations: static knowledge, inability to act, and lack of workflow management. AI agents address these challenges by integrating tools, performing actions, and orchestrating workflows.
• Reliable agents rely on key components: memory systems (both short-term for session context and persistent for long-term storage), tool integration to connect with external systems, and decision-making frameworks like ReAct and Tree-of-Thought for intelligent behavior.
• Memory comes in two main types: short-term memory for session-based context (implemented as a plain message list for full history retention) and persistent memory for cross-session information like user preferences. For longer conversations, an LLM-based summarization step can compress history to stay within context limits.
• The ReAct framework enables intelligent behavior through a continuous cycle of reasoning about next steps, executing actions, and observing results. This approach helps agents adapt dynamically to user needs while maintaining reliability.
• Agentic RAG systems merge retrieval capabilities with actionable insights, bridging the gap between knowledge retrieval and real-world tasks by combining document search with the ability to take actions.
• Advanced techniques such as tool validation, cross-referencing, and guardrails help prevent hallucinations and ensure accuracy, making agents more reliable for production use.

FAQ