Tool use extends language models beyond what is stored in their weights by letting them call external systems for fresh information, precise computation, or real-world actions. This is especially valuable for tasks a plain model cannot answer reliably, such as checking the current president or organizing files on a computer. The chapter distinguishes tool use in general from function calling, where tool arguments must follow a declared schema, and from code execution, where the “tool” is a program interpreter that can produce exact results.
The chapter explains that modern tool use emerged from earlier work on program-like neural systems, retrieval, web browsing, and model-program integrations, and has since grown into a central capability of advanced assistants and agents. Models may call search engines, calculators, calendars, APIs, databases, or code interpreters, often within multi-step loops of reasoning and action. In this interleaved pattern, the model reasons about what it needs, calls a tool, reads the result, reasons again, and repeats until it can answer confidently. Standards such as Model Context Protocol help make this interaction more uniform by defining predictable ways for applications to expose resources, prompts, and tools to models.
Practically, training tool-using models requires teaching them when to call tools, how to format calls correctly, and how to use returned outputs without treating them as if they were model-generated text. The chapter emphasizes implementation details such as schema validation, masking tool outputs from loss, managing multi-turn tool traces, and handling differences in API formats across providers. It also notes that tool-use data is usually synthetic or bootstrapped, with supervised fine-tuning providing a starting point, preference optimization improving decisions, and reinforcement learning becoming important for more complex agentic workflows where success depends on completing multi-step tasks in the environment.
Tool use interleaves model generation with external execution: the model generates tokens until it emits a tool call, an external system executes the tool and injects the output into the sequence, then the model continues generating. Models can emit multiple tool calls in a single generation. During training, tool call and output tokens are typically masked from the loss.
Summary
Tool use lets language models overcome fundamental limitations – knowledge cutoffs, imprecise arithmetic, lack of real-world interaction – by emitting structured requests that a system external to the model weights executes and injects back into the generation stream.
Training tool-use models follows the same post-training stages as other skills: SFT on tool trajectories teaches formatting and tool selection, preference optimization improves when to call tools, and RL with environment feedback trains multi-step agentic behaviors.
Model Context Protocol (MCP) has emerged as an open standard for connecting language models to external tools and data sources, providing a predictable JSON schema that decouples model vendors from tool implementations.
Implementation decisions include formatting the tool calls (Python vs. JSON), masking tool outputs from the training loss, handling reasoning token continuity across tool calls, constrained decoding for schema conformance, and managing context window consumption from tool outputs.
FAQ
What is tool use in the context of language models?Tool use is when a language model emits a structured request for an external capability, such as a search engine, calculator, database, or code interpreter. An orchestrator or host system executes the tool, appends the result back into the context, and the model continues generating. This extends the model beyond what is stored in its weights alone.Why do language models need tools?Tools help models handle tasks that are hard or impossible from weights alone, especially when the answer must be current, precise, or based on external systems. Examples include answering “Who is the president today?” or moving files in a local directory. Tools also help overcome structural weaknesses like outdated knowledge and probabilistic generation.What is the difference between tool use, function calling, and code execution?Tool use is the broad category: the model requests an external action and receives the output. Function calling is a form of tool use where the arguments must match a declared schema, usually JSON Schema, so they can be reliably parsed and validated. Code execution is a special case of tool use where the tool is a code interpreter, such as Python.How are models trained to interleave tool calls with generation?Training data typically includes a system prompt that lists available tools, along with examples of user queries, model tool calls, and tool outputs. The model learns when to emit a tool call, how to format the arguments, and how to use the returned result in its final response. During training, tool call and tool output tokens are often masked from the loss.What does the model do under the hood during a tool-use loop?The model generates text until it emits a tool call. Then an external system executes the tool and inserts the output into the message sequence. The model reads that output and continues generating, potentially repeating this cycle multiple times until it reaches an answer.What is interleaved thinking in multi-step tool reasoning?Interleaved thinking is the pattern of “think, call a tool, think again, call another tool” during one task. The model first reasons about what it needs, then gathers information with a tool, then reasons again using the new evidence. This approach supports more complex multi-step agent behavior and is associated with systems like ReAct-style reasoning.What is Model Context Protocol (MCP)?MCP is an open standard for connecting language models to external data sources and tools in a predictable JSON-based format. It uses JSON-RPC 2.0 and supports primitives such as resources, prompts, and tools. The goal is to standardize how applications expose context and actions to models.What are the main components of an MCP setup?An MCP server wraps a specific capability or data source, such as weather data or a database. MCP clients, like desktop apps or IDE plug-ins, aggregate one or more servers. The host application provides the user interface and model interface, making it easier to swap models or back-end tools without redesigning the whole system.What implementation details matter when building tool-use models?Important details include the chosen formatting style, such as JSON or Python; how tool outputs are masked during training; how multi-turn tool invocations are represented; and how tokenization and message templates are handled. Other concerns include reasoning-token continuity, provider-specific API formats, schema validation, and managing large tool outputs so they do not exhaust the context window.How are tool-use models evaluated and trained in practice?Evaluation can include exact match for tool names and arguments, schema validity, and end-to-end task completion. Reliability across repeated trials also matters, as in pass^k-style metrics that test consistency rather than one-off success. Training data is often synthetic or bootstrapped, and models are commonly improved with SFT, preference optimization like DPO, and RL with environment feedback for agentic tasks.
pro $24.99 per month
access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
Reinforcement Learning from Human Feedback ebook for free
