2 Building a conversational AI
This chapter presents a pragmatic, platform-agnostic path for building a production-ready conversational AI by walking through CakeBot, a helper for a small bakery. It starts from first principles—greeting, fallback behavior, and basic scaffolding—then incrementally adds capability while balancing control and flexibility. Although the examples use IBM’s watsonx Assistant and watsonx.ai, the design patterns are general: begin with narrowly scoped tasks, enforce clear responses where wording matters, and evolve toward richer automation and generative features as needs grow.
The first milestone is an intent-based FAQ bot. The team defines concise intents, trains each with a handful of example utterances, and crafts precise, copy-edited responses for predictable questions (background, hours, offerings, pricing, rewards). They tune fallback behavior to prevent user frustration and test iteratively. The chapter then extends static Q&A into dynamic responses by adding lightweight logic and clarification questions—illustrated with store hours that vary by location—showing how to collect disambiguating context, conditionally branch, and preserve user-provided details to keep conversations efficient.
Next, the bot evolves from answering to acting. It first serves as a routing agent, detecting intents like cake ordering and handing users to human specialists. Then it transitions toward a process-oriented flow that gathers key order details (e.g., size and occasion) before handoff, supporting “fall-through” when users volunteer information up front. Finally, the solution incorporates generative AI for open-ended queries—such as recipe requests—by capturing the whole user utterance, composing a targeted prompt, invoking an LLM via an API extension with tuned parameters, and returning the generated answer. The chapter’s throughline is an iterative blend of deterministic design, guided process automation, and selective LLM use to maximize coverage, quality, and control.
Default configuration of "no action matches" in the assistant.
Question-answering bots map user utterances to intents which map to answers.
User interface to create our first action.
Defining the utterances that trigger an action
Defining the response for a question-answering action. The simplest form has only one step after detecting the intent — give the response.
Chat preview link.
Example question-answer response from Cake Bot.
Process flow for a location-specific #store_hours intent.
The three steps for the #store_hours action.
Cake Shop’s cake order process as question-answering. But it doesn't really answer the question!
A routing agent detects user intents and then routes them to an appropriate specialist.
Routing agent configuration for #cake_orders. As soon as the intent is detected, the user is deflected to a human specialist.
Process flow for ordering a cake from Cake Shop.
Transitioning a routing agent to a process-oriented bot. The bot now collects two pieces of information before handing off to a human.
Step to collect cake size offers explicit choices to the user.
OpenAPI spec for our LLM text generation API with a subset of the possible request parameters.
Invoking an LLM text generation API from an action in the assistant.
Flow diagram for recipe generation in Cake Bot via LLM.
Displaying the output from the LLM call to the user.
Summary
- Question-answering (Q&A) bots are a great way to start building your first conversational AI.
- Training Q&A bots with examples of questions lets you give pre-defined answers to related groups of questions (intents).
- Actions start with an intent and can have many outcomes: answering a question, deflecting a user to a human agent, asking follow-up questions, and making API calls.
- A routing agent identifies intent and passes information to human agents. It is a great method to incrementally add capability to a conversational AI while leaning on human capability.
- Conversational AI can use a combination of traditional and rules-based techniques along with generative AI.
FAQ
What is an FAQ (question-answering) bot and when is it a good starting point?
An FAQ bot directly answers users’ questions with short, preauthored responses and minimal follow-up. It’s ideal when: - You have a small set of frequent questions. - You want strict control over answer wording. - You need to launch quickly and grow capabilities over time.How do intents, utterances, and responses work in intent-based Q&A?
- Utterances: Example phrasings users might say (e.g., “What are your hours?”).- Intent: The meaning group those utterances map to (e.g., #store_hours).
- Response: The controlled answer the bot returns for that intent. Modern systems can learn an intent from as few as ~5 sample utterances.
Which default actions should I configure first in a new assistant?
Prioritize: - Greeting: Clearly set expectations (what the bot can do). - “No action matches” (fallback intent): Define a helpful rephrase prompt and escalation path. - Trigger word handling: Decide how to manage profanity or other policy-sensitive inputs.How should I set up and tune the fallback (“no action matches”) behavior?
Best practices: - Count failed recognitions in a session. - After 1–2 misses, escalate to a human or alternate path to avoid user frustration. - Offer guidance to rephrase and suggest supported topics.What’s the difference between static and dynamic Q&A?
- Static Q&A: One fixed answer per intent (e.g., general cake pricing).- Dynamic Q&A: Answer varies based on context or user-provided details (e.g., store hours by location). Implement by collecting missing info, then tailoring the response.
Effective Conversational AI ebook for free
