8 Deploying agents and agentic systems
The chapter explains how agent systems move from demos into real applications by choosing the right way to consume and deploy them. It compares embedding an agent directly in a client app, exposing it as an API-backed service, or using one agent as a tool for another through protocols and agent-to-agent communication. The main idea is to match the deployment style to the job: embedded agents work well for fast, interactive experiences, while service-based and tool-based approaches fit longer-running or more complex workflows.
It then shows how containerization and orchestration make agent systems easier to manage at scale. Using Docker, agents can be packaged as microservices, upgraded independently, and run locally or in more scalable environments. Docker Compose extends this by letting multiple agent services work together as a single stack, and tunneling tools can temporarily expose local systems for demos and testing. The chapter emphasizes that a simple browser agent can remain responsive while delegating heavier work, such as image generation, to backend services.
The final part focuses on production concerns that become essential once agents are deployed for real users. It recommends picking the simplest runtime that satisfies latency needs, using clear communication paths, keeping state and memory disciplined, and designing idempotent tools for caching and replay. It also stresses release engineering, observability, reliability patterns, cost control, and strict security practices such as least privilege, secret management, sandboxing tools, prompt-injection defenses, and external policy enforcement. Overall, the chapter treats agents as software systems that need the same operational rigor as any other production service.
shows three simple patterns for deploying and consuming agents. From embedded agents, a microservice API is accessible or used as a tool through other agents.
Connecting to a real-time model using a RealTime Agent object in a web browser. Allows for vocal interaction with the agent hosted in the browser.
connecting the real-time voice agent to the API image generation agent as a tool and then generating images.
There are several ways afrontend agent may consume containerized microservice agents as tools through an API or as MCP servers.
Docker Desktop interface for managing containers, allowing a user to start/stop containers, delete containers, and images.
shows a set of containers orchestrated through a Docker Compose file.
illustrates how external tunneling options can expose locally running agent services to external users. The Actor represents an external network user accessing an agent service. First the user browses to the tunneling service address and then routed to the a developers local machine.
a helpful decision flowchart for deciding agent deployments.
The practical front-door agent deployment pattern used for user-facing agents and applications
Summary
- Agent consumption drives deployment: embed for ultra‑low latency UX, wrap as a synchronous API for request/response tasks, or run as event‑driven workers for long jobs and retries.
- Realtime agents in the browser (WebRTC/WebSocket) deliver barge‑in speech, token streaming, and the most responsive experiences—keep tools simple or proxy them server‑side.
- Microservices + containers cleanly separate concerns; agents make ideal microservices because they’re self‑contained and easy to scale, swap, and version.
- Dockerizing agent APIs standardizes runtime and dependencies; Compose lets you stand up multi‑agent stacks (UI, worker agents, tool services) with one command.
- External tunneling (e.g., localtunnel) turns local prototypes into shareable demos without full cloud deployment—useful for POCs and quick pilots.
- Choose the “wire” by latency and fit: WebRTC/WebSockets for realtime, HTTP+SSE for streamed request/response, and message buses for decoupled background work.
- Front‑door/orchestrator patterns route user intents to specialized worker agents; keep the front‑door light and push complexity into typed, well‑scoped workers.
- State and idempotency matter: store short‑term chat state separately from long‑term knowledge, and make tool calls idempotent to enable caching, replay, and resilience.
- Release engineering applies to agents: version prompts, tools, and models; promote with gates; pin exact model/tool versions for reproducibility and incident debugging.
- Observability is non‑negotiable: trace from UI → gateway → agent → tools → model; track latency, cost, and success metrics; prefer structured logs with PII redaction.
- Reliability patterns—timeouts, fallbacks, circuit breakers, and graceful degradation—keep systems useful even when tools or models misbehave.
- Cost control comes from routing by intent, trimming context, and caching deterministic results—lower tokens often means lower latency, too.
- Security, safety, and governance must be built‑in: threat‑model surfaces, enforce least privilege, manage secrets correctly, sandbox tools, and defend against prompt‑injection with schema‑first tool contracts and instruction hierarchies.
- With deployment patterns, observability, and safety in place, agents graduate from demos to dependable, production‑ready systems.
AI Agents in Action, Second Edition ebook for free
