7 Augmenting intent data with generative AI
Users get frustrated when bots fail to understand them; this chapter proposes a hybrid approach that uses generative AI to augment intent data while keeping a purpose-built conversational AI stack in control. It contrasts LLMs (powerful and general but slower, costlier, less explainable, and prone to hallucinations) with traditional NLP (fast, cheap, controlled, but narrower) and shows how combining them yields the best of both. LLM augmentation shines when bootstrapping from no data, expanding sparse or rare-intent data, and creating robust test sets. To do this safely, teams pair domain knowledge with careful model selection, weighing terms of use, data privacy, capability, openness, and latency.
In practice, the human–LLM partnership is iterative: prompt to surface contextual noun/verb synonyms, then generate full utterances that span diverse grammatical forms (questions, statements, fragments, passive voice, qualifiers), adjusting decoding for creativity. One-shot and few-shot examples steer style, and trying multiple models seeds different ideas. Builders then handpick a compact, high-variation training set that generalizes across many low-variation phrasings, while reserving a larger, diverse test set the bot was not trained on. Attempts to do everything in one prompt fall short; human review curates coverage, reduces hallucinations, and transfers lessons to new intents (for example, evolving from “forgot password” to “store location”).
Beyond utterance generation, LLMs can brainstorm missing intents and sanity-check intent boundaries by classifying example utterances; any disagreement exposes potential confusion or over-fragmentation. Template-based mutations and the addition of greetings or closures can systematically grow test data, but should not dominate training to avoid bias. The chapter’s guidance distills to clear dos and don’ts: treat the LLM as a creative partner, provide context and examples, experiment widely, and always review outputs; don’t expect a single prompt to do everything or share sensitive data with models that retain it. Used at build time under human oversight, generative augmentation speeds up intent design, strengthens understanding, and improves robustness without sacrificing cost control or response reliability.
Visualization of coverage from different kinds of utterance sets. Our ideal training data is set 4 that covers large variation in small number of utterances. Set 3 is the ideal testing data.
Generating additional examples from initial LLM output. The LLM generated "My password isn't working" but we now know a related utterance is "My login information isn't working"
Converting "I forgot my password" into a template that lets us replace verbs and nouns in context. One option from this template is “I lost my credentials”.
Using common templates to see whether greetings and closures affect the chatbot’s understanding. One possible utterance is “Hello I lost my credentials please assist.”
A large language model augments the human builder in many ways.
Summary
- LLMs are a great partner that augment human builders. Humans and LLMs are better together.
- Experiment with different models, prompts, and parameters to get the best output from LLMs. Keep iterating! Don’t expect your first attempt to be perfect.
- Don’t just give instructions to an LLM. Provide examples through one-shot or few-shot prompts.
- When you identify a gap in your data, you can ask an LLM to help you fill it.
- LLM output can be used directly in your training data or you can manually refine it first.
- Use greedy decoding to get the same output every time. Use sampling decoding to get randomized responses with additional creativity. Execute the same prompt multiple times with sampling decoding to get a variety of responses and use the most helpful output.
FAQ
What does “augmenting intent data with generative AI” mean, and why use a hybrid approach?
It means using an LLM as a creative partner to help a human builder produce and refine intent training and testing data. The hybrid approach keeps a fast, low-cost, controlled classifier in production while leveraging LLMs at build time to expand coverage, speed up data creation, and reduce missed user expressions—without handing full control (and hallucination risk) to the LLM.When should I use an LLM to generate utterances for my chatbot?
Use it to 1) bootstrap when you have no production data, 2) expand sparse data sets (especially for rare but important intents like fraud), and 3) create robust test sets that better probe your model’s understanding, even if your runtime solution uses RAG or rules.How do LLMs compare to traditional NLP classifiers for intent detection?
Traditional classifiers are purpose-built, fast, low-cost, and controllable but can struggle to scale beyond ~100 intents and need curated data. LLMs are powerful, general, and often accurate but are slower, costlier, less explainable, and can hallucinate. A hybrid uses the best of both: LLMs at build time; traditional or RAG-driven systems at runtime.What should I consider when selecting an LLM to augment my data?
Check: 1) terms and conditions (some forbid “building/improving another model”), 2) data privacy (will it train on your data?), 3) capability (instruction following, creativity), 4) openness (open-source transparency vs proprietary ease), and 5) latency/cost trade-offs.How should I use LLM-generated data safely and effectively?
Use it at build time to: add training examples, add test cases, and mutate existing utterances (e.g., synonyms, grammar). Prefer small, high-variation training sets and keep LLM outputs under human review. Test on data not seen in training.How can I generate high-variation utterances with an LLM?
- Start with contextual synonyms for key nouns (e.g., “password” → “login information,” “credentials”) and verbs/verb phrases (e.g., “forgot” → “can’t remember,” “lost”).- Generate full utterances with varied grammar: statements, questions, commands, fragments, active/passive, qualifiers.
- Use one-shot/few-shot examples to steer format; adjust decoding (sampling/temperature) for creativity; iterate.
Why are high-variation utterances better than many low-variation ones?
High-variation sets cover more linguistic space with fewer examples (e.g., “I can’t log in” covers cases that don’t share words with “forgot password”). It improves robustness and reduces the risk of overfitting to one phrasing pattern. Ten strong, varied examples often beat 100 near-duplicates.What’s the role of templates, and what are their pitfalls?
Templates are great for rapidly generating many controlled test utterances (e.g., swapping verbs/nouns; adding greetings/closures to test distraction tolerance). Don’t overuse them for training—they can create imbalance and overemphasize one grammatical pattern.Can I transfer prompting lessons across intents?
Yes. Use one-shot or few-shot prompts that show desired grammatical structures from one intent (e.g., password reset) and ask for analogous phrases in another (e.g., store location). Expect to iterate; a single “do everything” prompt often misses some instructions.How can an LLM help me find gaps or confusion in my intents?
- Brainstorm additional intents when historical data is scarce.- Ask the LLM to cluster or label your training utterances; mismatches can reveal overlap or missing boundaries (e.g., “login” vs “password reset”).
- Treat outputs as guidance, not ground truth; review against your bot’s design and the Do/Don’t best practices.
Effective Conversational AI ebook for free
