2 The brain of AI agents: LLMs

At the start of building an AI agent, the chapter frames the language model as the agent’s “brain” and asks the core practical questions that shape development: which model to choose, how to call it reliably, and where its abilities stop. It uses the example of a Research Agent to show that the model must do more than generate text; it must interpret requests, reason over information, and support decision-making in a way that can be turned into useful actions.

The chapter then explains how to choose between closed and open models, recommending closed LLMs first because they are easier to prototype with, offer stronger performance, and already include production-friendly features such as tool calling, structured output, and cost-saving optimizations. It shows how APIs are used through OpenAI-style chat completions and how LiteLLM provides a single interface for many providers, reducing vendor lock-in. It also emphasizes essential agent-facing API skills like managing conversation history in stateless calls, enforcing structured JSON-like outputs with schemas, and using asynchronous requests with rate limiting and retries to handle many calls efficiently.

Finally, the chapter focuses on prompt engineering and a benchmark-based reality check. It argues that system prompts should define the agent’s identity, boundaries, tone, and knowledge limits, while also guiding autonomous action: when intent is clear, the agent should act immediately; when it is ambiguous, it should ask only minimal questions. It then uses GAIA to test models on realistic tasks and shows that even strong LLMs solve only a small portion without external tools, often due to current-information needs, file access, or unreliable self-assessment. The takeaway is that LLMs are powerful but incomplete, and effective agents must combine them with tools and clear behavioral design.