Overview

4 The LLMAgent Class

This chapter introduces the first complete LLM agent in the framework by combining the backbone LLM, tools, prompt templates, and task-oriented data structures into a working system. It begins by defining the agent’s core responsibilities: receiving a task, planning how to solve it, and invoking tools when needed. To support this, the chapter adds simple structured models for tasks and task results, along with matching step-level structures that will later drive the agent’s internal execution flow.

The middle of the chapter focuses on the agent’s architecture and the processing loop that powers its behavior. The agent class stores the LLM, a tool registry for fast lookup, and templates used to standardize prompts. A nested task handler manages execution as a sequence of discrete sub-steps: it asks the LLM what to do next, runs one step at a time, executes any requested tool calls, and records a rollout history of the interaction. The design uses structured outputs to decide whether to continue with another step or return a final result, and it wraps the whole process in an awaitable future so task execution can run concurrently in the background.

The chapter closes by demonstrating the agent on a Hailstone-sequence example, showing how the processing loop coordinates repeated tool use until the task is complete. The example highlights why prompt design matters, including the need to hide obvious cues so the model actually uses the tool rather than relying on memorized knowledge. Finally, the chapter connects this agent design to larger multi-agent systems by showing how the same mechanism can support specialized subagents, delegated work, and remote agents in future framework extensions.

Having defined classes for tools and LLMs in our framework, we can now complete part 1 by building the LLM agent.
Having defined classes for tools and LLMs in our framework, we can now complete part 1 by building the LLM agent.
Our LLM agent, now showing the LLMAgent class with its core attributes and run() method.
Our LLM agent, now showing the LLMAgent class with its core attributes and run() method.
Submitting Task objects to the run() method of an LLMAgent produces a TaskResult.
Submitting Task objects to the run() method of an LLMAgent produces a TaskResult.
UML class diagrams for LLMAgent and the new data structure classes, Task and TaskResult, that it works with to perform tasks.
UML class diagrams for LLMAgent and the new data structure classes, Task and TaskResult, that it works with to perform tasks.
Templates are customizable text structures that help to standardize interactions with an LLM.
Templates are customizable text structures that help to standardize interactions with an LLM.
Executing a task through sub-steps within a processing loop. Each sub-step uses the LLM's planning and tool-calling capabilities, with the results informing the next sub-step's instruction.
Executing a task through sub-steps within a processing loop. Each sub-step uses the LLM's planning and tool-calling capabilities, with the results informing the next sub-step's instruction.
The execution of sub-steps within a processing loop using the data structures TaskStep, TaskStepResult and NextStepDecision
The execution of sub-steps within a processing loop using the data structures TaskStep, TaskStepResult and NextStepDecision
UML class diagrams for TaskStep, TaskStepResult, NextStepDecision, and TaskHandler
UML class diagrams for TaskStep, TaskStepResult, NextStepDecision, and TaskHandler
The process for executing tasks using the run() method. When a task is submitted, the helper TaskHandler is created to manage task execution on behalf of the LLMAgent.
The process for executing tasks using the run() method. When a task is submitted, the helper TaskHandler is created to manage task execution on behalf of the LLMAgent.
Adding new entries to the rollout attribute after every sub-step during task execution
Adding new entries to the rollout attribute after every sub-step during task execution
The get_next_step() execution to produce either a TaskStep or TaskResult
The get_next_step() execution to produce either a TaskStep or TaskResult
The run_step() execution using a single-turn LLM chat interaction between the LLMAgent and its backbone LLM
The run_step() execution using a single-turn LLM chat interaction between the LLMAgent and its backbone LLM

Summary

  • The LLMAgent class creates LLM agents from an LLM and a set of Tool objects.
  • The Task data structure contains an instruction for an LLMAgent to execute with the run() method. Awaiting the run() method returns a TaskResult data structure.
  • Executing a task within the run() method involves various interactions with the backbone LLM. To standardize our interactions, we use prompt templates defined by the LLMAgentTemplates type.
  • The processing loop, where task execution occurs, is performed through a sequence of discrete sub-steps. Implementations of the processing loop, such as the ReAct processing loop, differ in their definitions of sub-steps and the work performed within them.
  • Our processing loop implementation is simple, involving one planning and one tool-calling phase. Between each sub-step, the LLMAgent uses its backbone LLM to determine whether a new sub-step is required or if the overall task is complete.
  • The data structures TaskStep, TaskStepResult, and GetNextDecision help to facilitate the implementation of our processing loop.
  • The TaskHandler is a helper class, nested within the LLMAgent class. It’s designed to orchestrate the execution of a submitted Task through the discrete sub-steps of our processing loop.
  • TaskHandler inherits from asyncio.Future, ultimately resolving to a TaskResult. It also supports two methods get_next_step() and run_step(), that are executed in tandem within the processing loop until task completion.
  • The run() method creates a TaskHandler and schedules the processing loop for execution as a background asyncio.Task, then returns the TaskHandler for flexible concurrency.
  • When task execution completes, the result of the TaskHandler is set to the TaskResult. As a subclass of asyncio.Future, the TaskHandler and thus also the run() method can be awaited.
  • Rollouts, otherwise known as trajectories, provide a useful log of the LLM agent’s actions and reasoning during task execution. The information they provide can be used to evaluate and improve an LLM agent.

FAQ

What is the main purpose of the LLMAgent class in this chapter?The LLMAgent class is the core agent that combines a backbone LLM with a set of tools so it can execute user-submitted tasks. Its job is to use the LLM’s planning and tool-calling abilities to complete tasks through a processing loop.
What data structures are introduced to represent tasks and their final results?The chapter introduces Task and TaskResult. Task stores a unique id_ and an instruction, while TaskResult stores the task_id of the task it came from and the resulting content.
Why does TaskResult include a task_id field?TaskResult includes task_id so it can be linked back to the Task that produced it. This makes it easy to trace results to their originating task.
What is the role of the tools_registry attribute in LLMAgent?tools_registry provides fast lookup of tools by name. It is used when the agent needs to resolve and execute a tool-call request made by the LLM.
Why does LLMAgent use prompt templates?Prompt templates standardize the way the agent communicates with its backbone LLM. They let the agent fill in placeholders with task-specific details, rollout history, and step instructions.
What is the add_tool() method used for?The add_tool() method registers a new tool with the LLMAgent. It checks for duplicate tool names first and raises an LLMAgentError if a name conflict is found.
What is TaskHandler and why is it needed?TaskHandler is an inner helper class of LLMAgent that manages task execution. It handles the processing loop, step execution, background task management, and eventually resolves to a TaskResult.
What are TaskStep, TaskStepResult, and NextStepDecision used for?TaskStep represents a single sub-step instruction, TaskStepResult stores the outcome of that sub-step, and NextStepDecision is a structured output used to decide whether the agent should continue with another step or return the final result.
How does the run() method execute a task?run() creates a TaskHandler and starts an async background processing loop. That loop repeatedly calls get_next_step() and run_step() until the task finishes, reaches a stopping condition, or raises an error.
What is a rollout in LLMAgent, and why is it useful?A rollout is a formatted log of the sub-steps taken during task execution, also called a trajectory. It helps you inspect how the agent reasoned, what tools it called, and how it arrived at the final result.

pro $24.99 per month

  • access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
  • choose one free eBook per month to keep
  • exclusive 50% discount on all purchases
  • renews monthly, pause or cancel renewal anytime

lite $19.99 per month

  • access to all Manning books, including MEAPs!

team

5, 10 or 20 seats+ for your team - learn more


Close
choose your plan

team

monthly
annual
$49.99
$499.99
only $41.67 per month
  • five seats for your team
  • access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
  • choose another free product every time you renew
  • choose twelve free products per year
  • exclusive 50% discount on all purchases
  • renews monthly, pause or cancel renewal anytime
  • renews annually, pause or cancel renewal anytime
  • Build a Multi-Agent System (from Scratch) Build a Multi-Agent System (from Scratch) ebook for free
Close
choose your plan

team

monthly
annual
$49.99
$499.99
only $41.67 per month
  • five seats for your team
  • access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
  • choose another free product every time you renew
  • choose twelve free products per year
  • exclusive 50% discount on all purchases
  • renews monthly, pause or cancel renewal anytime
  • renews annually, pause or cancel renewal anytime
  • Build a Multi-Agent System (from Scratch) Build a Multi-Agent System (from Scratch) ebook for free