9 Automating workflows with agentic AI
This chapter introduces agentic AI—systems that combine language models with the ability to act in the world—to automate end-to-end workflows. Building on predictive and generative AI, agents use natural language to understand tasks, choose tools, and execute actions such as research, analysis, and communication, enabling significant efficiency gains across domains like product management, marketing, and sales. The author stresses pragmatic adoption: start with constrained, high-coverage use cases, keep humans in the loop, and design the user experience for transparency and trust. Early production patterns include 80–20 automation and dogfooding, with automation expanding as reliability improves and user comfort with AI grows.
The chapter details how agents operate: they access diverse tools to collect data, perform analyses beyond LM capabilities, and take “write” actions with safeguards. Best practices include tight tool descriptions, minimal and clear interfaces, careful error handling for tool selection, calling, and output parsing, and treating the human as a tool for approvals, preferences, and missing inputs. Architecturally, the LM acts as the agent’s brain, governed by prompts that encode goals, policies, tools, planning guidance, and memory. Planning is enabled via stepwise reasoning and reflection (e.g., decomposition and iterative refinement), often reinforced by explicit, scripted flows for robustness. Memory spans short-term context management and long-term stores for personalization, task outcomes, and reusable insights, supporting continuous learning. Risk-aware design, guardrails, and HITL checkpoints are emphasized for any action that changes external systems.
Operating at the frontier brings constraints: finite context windows, brittle long-horizon planning, the unpredictability of language interfaces to software, and latency. To mitigate these, the chapter advocates incremental, modular development; rigorous monitoring and evaluation; conservative tool counts; explicit safety policies; and a thoughtful balance between framework convenience and custom implementations. Looking ahead, it explores multi-agent collaboration (specialized agents coordinated by a supervisor), conversational access to organizational data (combining retrieval with structured querying), and the longer-term vision of the autonomous enterprise that automates, self-optimizes, and adapts across business functions. Realizing this vision requires gradual rollout, strong UX for collaboration, robust security and auditability, and sustained human oversight.
Comparing a human workflow with an LM agent
Example prompt for an agent for PM tasks
Template for a product roadmap
Extending the agent with multiple categories of tools
As the agent becomes more reliable, you can reduce human involvement, simplifying your user interface.
ChatGPT plug-ins (https://openai.com/blog/chatgpt-plugins)
The agent’s process for using a tool
High-level architecture of an agent
An intelligent agent interacts with an external environment.
Prompt template for a roadmapping agent
Agents can use different planning methods
Plan for a product roadmap update (without reflection)
Reflecting on the feedback for a subtask
Agents need a short-term memory for the current session, and a long-term memory for learning and improvement over time.
The “supervisor” pattern in agent collaboration: an overarching agent coordinates the execution of subtasks by specialized agents
Summary
- An intelligent agent is an autonomous system that acts in and on an environment to maximize some reward from its actions.
- An LM-driven agent has four components - the controlling language model, a range of external tools, a memory module, and a planning module.
- Function calling and tool access extend the capabilities of your agent by integrating external tools for tasks the agent can’t handle natively.
- Planning and memory modules allow agents to reason, reflect, and learn over time, improving their effectiveness in complex, multi-step tasks.
- Start with a lean setup and grow in small steps. Initially, your agent can just be a detailed prompt describing its main task and specifying a couple of tools. Over time, you can add more complexity to each of its components.
- When adopting agent frameworks, evaluate the trade-off between flexibility and initial convenience, and be cautious of high switching costs and overhead; sometimes, building custom solutions may be more effective.
- Use human-in-the-loop (HITL) processes to support the agent when confidence is low or catastrophic errors can happen, ensuring quality control and reducing the risk of wrong decisions or actions.
- Incrementally improve your agent's performance by focusing on manageable tasks and gradually increasing the scope as confidence in the system grows.
- Address limitations of agent systems, such as finite context length, unpredictable outputs, and task decomposition issues, with pragmatic strategies to manage risks.
- Prepare for rising opportunities like multi-agent collaboration and autonomous enterprises, while using the current challenges as stepping stones for developing more advanced systems.
FAQ
What is agentic AI, and how does it differ from predictive and generative AI?
Agentic AI adds the ability to act. Predictive AI finds patterns in existing data; generative AI produces new content. Agentic AI uses a language model to plan, choose tools, and execute actions in the external world (e.g., search the web, query databases, send emails), often with iterative reasoning and feedback.When does it make sense to ship an agent to production?
- 80–20 scenarios: automate the easy 80–90% and route the hard cases to humans (e.g., customer service agents).- Dogfooding: first deploy to AI-savvy internal teams or early adopters who accept uncertainty (e.g., coding agents).
Start small, measure quality/latency, and expand as reliability improves.
How do language models access external tools?
Via function calling: the LM selects a described function/API and provides structured inputs; the system executes the call and returns results. You can use plugin ecosystems or frameworks (e.g., LangChain, LlamaIndex) and, if needed, fine-tune models or use function-calling–optimized LMs. Thoroughly test third-party tools for reliability and safety.What tool categories are useful for automating a product roadmapping workflow?
- Data collection: connectors to sales calls, interviews, usage data, reviews, CI sources, web search; use RAG for unstructured data and Text2SQL for structured data.- Analysis: calculators, rules, and ML models (e.g., sentiment/topic analysis, segmentation).
- Action/communication: email/Slack, updating artifacts (e.g., roadmap docs).
Start lean, then add sources/tools as confidence grows.
What are the risks of giving an agent “write” access, and how can I mitigate them?
- Require human confirmation for high-stakes actions (thresholds, multi-step verification).- Add guardrails: allow/deny lists, scope limits, and role-based access.
- Implement rollback and audit logs.
- Simulate and red-team failure modes before enabling write access.
How do I integrate Human-in-the-Loop (HITL) without overwhelming users?
Treat the human as a tool the agent can call for: low-confidence reviews, subjective preferences, and missing data. Choose the right UX mode—chat, background agent with dashboards, or collaborative co-editing—and reduce human touchpoints as reliability improves. Consider users’ AI affinity when designing interventions.What does a minimal agent architecture look like, and what goes into the prompt?
A lean setup uses an LM as the “brain” plus a small set of tools. The agent prompt typically includes: job description/goals, HITL policy, available tools (with clear descriptions), planning guidance, relevant memory, and the current request. Add style/persona only if it helps outcomes. You can defer separate planning and memory modules until needed.How should agents plan tasks effectively?
- Use Chain-of-Thought/Tree-of-Thought to decompose tasks step by step.- Add reflection (e.g., Self-Refine) to iteratively critique and improve outputs.
- For reliability, hard-code stable parts of workflows (explicit flow engineering).
- Include risk-aware checks for potentially catastrophic actions.
How do agents use memory (short-term vs. long-term)?
- Short-term: keep key steps/results in the LM context (summaries over raw traces) to guide multi-step reasoning; manage context size to control cost/latency.- Long-term: store interactions, summaries, embeddings, and reflections in external stores (e.g., vector DBs) for retrieval and learning over time. Systems like MemGPT/Letta illustrate virtual context management with self-editing memory.
What are common challenges and practical ways to address them?
- Constraints: limited context, brittle planning, unpredictable NL I/O with tools, and latency.- Tactics: limit tools (≈4–5), provide precise tool descriptions, use few-shot examples, semantic tool retrieval, heuristics, or fine-tuning for tool use; simplify input/output schemas; parse and trim API outputs.
- Process: iterate incrementally, monitor SOTA, choose frameworks judiciously (avoid heavy abstraction lock-in), and expand to multi-agent setups carefully. Emerging opportunities include multi-agent collaboration, conversational data access, and steps toward autonomous enterprises.
The Art of AI Product Development ebook for free
