11 Tips for building agentic systems

This chapter shifts from theory to practical building advice for production agentic systems, using the five agent layers as a framework for design. The main message is that strong agents come from disciplined choices at every layer: keep the persona sharply defined, make tools focused and reliable, control reasoning loops, ground answers in retrieval, and instrument everything so the system can be evaluated and improved over time.

For the core persona, the chapter emphasizes treating instructions like an API contract: set clear role boundaries, keep the scope narrow, prefer structured outputs, and inject runtime facts only when they matter. For tools and actions, it recommends single-purpose tools with clear schemas, using function-based tools when possible, constraining when tools can run, relying on trusted prebuilt capabilities where appropriate, and designing for failure with retries and graceful fallbacks. Across both areas, the recurring theme is to reduce ambiguity, token bloat, and unintended behavior.

The later sections apply those principles to common agent patterns. For reasoning and planning, the chapter favors explicit plan-then-execute loops, ReAct-style iteration, turn limits, and self-review, while warning against uncontrolled reasoning in simple tasks. For knowledge and memory, it frames retrieval as the foundation of useful context, advocating RAG-first designs, selective long-term memory, hybrid search, careful chunking, and strict grounding. For evaluation and feedback, it stresses tracing, automated tests, human feedback, guardrails, and DevOps-style iteration. These ideas are then adapted to customer support, RAG workflows, and deep research systems, where modular agents, escalation paths, grounding discipline, and critic/writer-style oversight help deliver answers that are safe, accurate, and useful.

Highlights the five agent layers and for each layer the internal boxes represent the patterns that can help improve development of the respective layer.

shows an agent deployment wrapped with a best practices evaluation and feedback system

demonstrates an entire customer support agentic system workflow.

demonstrates a more advanced RAG agent system that can query from multiple different context sources to answer a question/search.

illustrates the implementation of the deep research agent system.

• Treat agent design as five stacked layers: persona, tools/actions, reasoning & planning, knowledge & memory, and evaluation & feedback. Optimize each layer independently, then integrate.
• Write the persona as an API contract, not prose: clear role, boundaries, and an explicit “I don’t know” off‑ramp.
• Keep scope narrow (single responsibility) for higher accuracy and easier testing/cost control.
• Prefer structured, typed outputs (Pydantic/dataclass JSON) to eliminate parsing brittleness.
• Use dynamic instructions (inject date/org/user) at run time to avoid stale facts.
• Build single‑responsibility tools with crisp names, docstrings, and typed params; the model learns tool use from these schemas.
• Control when tools run (e.g., tool_choice, parallel tool calls); stop early when one call suffices.
• Leverage pre-built/hosted tools where sensible, but be selective.
• Plan for failures: timeouts, retries, graceful error payloads, and fallbacks.
• Adopt ReAct (reason → act → observe) or Chain‑of‑Thought patterns for non-trivial tasks.
• Plan then execute draft a short checklist before acting; consider a lightweight “thinking” server/step.
• Cap iterations to prevent runaway loops; signal needs_followup on budget/turn limits.
• Add a quick self-review pass before finalizing to catch obvious mistakes.
• Treat RAG as first‑class: use a retrieval tool over a KB instead of stuffing docs into prompts.
• Separate session memory (short-term thread context) from long-term memory (vector/DB); prune aggressively.
• Optimize retrieval: ANN indexes, shard by domain, filter with metadata, consider domain/multilingual embeddings.
• Use hybrid search (semantic + keyword + hierarchical/graph) for recall/precision.
• Chunk semantically, ground answers strictly in the retrieved context, and be comfortable saying “don’t know.”
• Keep data fresh and partitioned by role/tenant to enforce access boundaries.
• Trace everything: prompts, tool calls, token counts, latency, outcomes; wire to your observability stack.
• Automate evaluations (LLM‑as‑judge + curated test sets) on each prompt/model change.
• Incorporate HITL: collect user feedback, review complex cases.
• Add guardrails & moderation at inputs, tool calls, and outputs; validate schemas.
• Practice AIOps: measure → observe → iterate, including downstream systems (indexes/APIs) your agents depend on.
• Narrow the charter (orders, returns, status) first; expand gradually.
• Ground every answer via RAG; add a dedicated grounding agent if needed.
• Provide HITL escalation (escalate_to_human) and capture user feedback.
• Enforce identity & least‑privilege access before revealing account data or invoking business APIs.
• Engineer resilience (retries, fallbacks, timeouts) and cache popular answers; apply rate limits.
• Maintain transparent traces per conversation for auditability.
• Real‑world “support agent” = multiple cooperating agents (triage → retrieval → grounding → action/guardrail → answer).
• Use an agentic orchestrator only when needed; if one-shot RAG suffices, keep it simple.
• Prefer modular roles: router/triage → retriever → answerer; optionally a Critic (CRAG) for corrective retrieval.
• Optimize retrieval (ANN, filters, rerankers, HyDE) and enforce grounding with citations.
• Instrument recall, accuracy, and “should have said I don’t know” cases; close the loop with evals.
• For advanced systems, support multiple corpora, ranking/refinement loops, and bounded retries.
• Use two-tier orchestration: a Planner (brain) delegates to stateless Workers (hands) for web search, extraction, analysis, and writing.
• Enforce tool‑use policies: no claims without cited sources; facts must come from retrieval/doc tools.
• Add self-critique/fact‑check before the final report (Critic pass).
• Provide a streaming UX to surface partial findings and outlines as they emerge.
• Manage caching & budgets: dedupe repeated queries, cap tokens/calls, and clear caches periodically to avoid staleness.
• Production-ready agentic systems emerge from tight personas, well-scoped tools, guided reasoning, disciplined retrieval, and relentless measurement—assembled into modular, auditable workflows with clear escalation paths and grounded outputs.

FAQ