Prompt engineering is presented as a practical, fast way to steer language models without retraining, useful for both technical and non-technical product teams. The chapter follows Alex as he graduates from simple zero-shot prompts to richer structures that combine clear instructions with context, examples, constraints, and explicit output formats. It emphasizes modular prompt components and reusable templates to accelerate iteration, shows how few-shot prompts enable learning by analogy, and flags biases that creep in through example selection (majority label, recency, and common token effects). To scale quality and control cost, it recommends curating compact example databases and automatically retrieving the most relevant demonstrations per input.
To help models reason and act on complex tasks, the chapter introduces chain-of-thought to decompose problems, prompt chaining to focus attention step by step, self-consistency to generate multiple candidates and select the best, and reflection to critique and improve drafts. These techniques leverage inference-time scaling—giving the model more “thinking” tokens—to boost reliability on multi-step problems while noting the cost trade-offs and diminishing returns on simple queries. Used together, they drive more coherent, accurate outputs for tasks like long-form content, analysis, and planning, often approaching fine-tuned performance without new training.
Finally, the chapter systematizes prompt engineering as an ongoing process: be precise, provide rich context, and specify structured output formats; explore multiple perspectives for diversity; document experiments, version prompts, and collaborate across teams; and rigorously evaluate with test suites that include edge cases. It advocates production-grade practices—monitoring real user inputs, adding guardrails, compressing prompts to save tokens, and exploring automated prompt tuning—to sustain quality at scale. Throughout, prompts are positioned as a durable asset that underpins more advanced architectures such as retrieval-augmented generation, fine-tuning, and agentic systems as products mature.
Overview of the most popular prompting techniques
Example of an input-output prompt for sentiment analysis
Example of a few-shot prompt for sentiment analysis
Workflow of an automated system for few-shot prompting
Schema for chain-of-thought prompting
Schema of self-consistency prompting.
Double diamond model of a creative process
Summary
Start with simple zero-shot prompts for straightforward tasks or well-covered topics.
Use few-shot prompts by providing examples when dealing with complex tasks.
Break prompts into reusable components like context, instructions, and examples to save time.
Approach prompt engineering with structure and an iterative improvement process.
Use chain-of-thought prompting to guide the model through tasks that require multiple steps.
Use self-consistency to run multiple prompt versions and choose the best output for creative tasks.
Use reflection prompts to have the model review and improve its output.
Compress context and examples to optimize prompts and reduce token consumption.
Continuously monitor and adjust prompts based on real-world user interactions.
Use tools and version control to manage and refine your prompt development efficiently.
FAQ
How does zero-shot prompting work, and when is it sufficient?Zero-shot prompting provides only the task/instruction (and optional brief context) and lets the model generate the next tokens based on its pretraining. It works well for simple or common tasks seen in training data. To sharpen results, add concise context, style, and constraints. For complex or novel tasks, prefer few-shot or reasoning techniques like chain-of-thought.What are the core components of an effective prompt?The typical components are:
- Context/role: domain info, persona, prior dialogue
- Instruction: the task to perform
- Examples: demonstrations of input-output pairs
- Input variables: placeholders (topic, style, audience)
- Output format: structure and data format (e.g., JSON)
- Constraints: limits on length, style, vocabulary
Choose only what’s needed to reduce tokens and latency.How do I create and use prompt templates without losing flexibility?Break prompts into modular components with variables (e.g., {topic}, {content type}, {style}). Store successful structures as templates and fill variables at runtime using a templating tool (e.g., LangChain). Start manual, then introduce tools; premature tooling can constrain exploration. Document iterations so winning patterns are reusable.What is few-shot prompting, and what trade-offs should I expect?Few-shot prompting adds high-quality demonstrations to teach by analogy. It often boosts performance over zero-shot for short or unfamiliar tasks, but:
- Increases token usage and latency
- Requires effort to curate examples
- Can hit context-length limits
Review and prune “inflated” prompts; sometimes zero-shot works better for long-form content.How can I recognize and mitigate biases in few-shot examples?Common biases:
- Majority label bias: unbalanced labels skew outputs; balance your examples.
- Recency bias: last examples weigh more; avoid ending on the dominant label.
- Common token bias: frequent labels appear more; mildly overrepresent rare labels.
Version and test different example orders to observe and correct drift.How do I automate few-shot prompting with an example database and retrieval?Build a curated, diverse example set (small but high-quality). At inference, retrieve the most semantically similar examples using embeddings and a vector store. For classification tasks, “lazy” random labeling can still convey the label space and output format, though correctness of labels matters less than coverage. Keep an eye on token limits.What is chain-of-thought (CoT) prompting, and when should I use prompt chaining?CoT has the model decompose a problem and “think step-by-step,” either with explicit subtasks or by asking it to think aloud. It improves multi-step reasoning but adds tokens/cost. Prompt chaining runs each subtask in a separate prompt, feeding intermediate results forward, improving focus and accuracy. Use for complex tasks; skip for simple fact recall.What is self-consistency prompting and how do I implement it?Self-consistency samples multiple answers (raise temperature), then selects the best:
- Categorical tasks: majority vote
- Open-ended tasks: self-evaluation/scoring or external checks (e.g., unit tests for code)
Combine with CoT and chaining. It’s especially useful for creativity (diverge → converge) but increases compute.How does reflection-based prompting improve outputs, and what are its limitations?Reflection asks the model to critique and revise its own draft, optionally in two steps: (1) evaluate against criteria and propose fixes, (2) apply fixes. It’s powerful for complex, knowledge-heavy writing. Limitations: if the first draft is fundamentally wrong, reflection may amplify errors; it can also add verbosity. Guide with clear criteria and output formats.What are best practices for systematizing, optimizing, and evaluating prompts in production?- Know your model’s capabilities/limits; iterate empirically.
- Be precise; provide rich but concise context; specify output formats (e.g., JSON).
- Collaborate across roles; manage prompts with templates and version control (e.g., LangChain, PromptAppGPT, Prompt Engine).
- Optimize tokens via prompt compression; consider parameter-efficient prompt tuning.
- Evaluate with test suites (relevance, coherence, accuracy) and edge cases; use independent checks.
- Monitor real user inputs for domain shift; add guardrails (e.g., NeMo, Guardrails AI).
- Use inference-time scaling (CoT, self-consistency) when the task needs deeper reasoning; weigh cost vs. benefit.
pro $24.99 per month
access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
The Art of AI Product Development ebook for free
