1 Introduction to prompt programming and DSPy
The chapter introduces the early, often frustrating stage of building LLM applications, where small prompt changes can lead to very different results and developers spend significant time guessing, testing, and retesting. It contrasts traditional prompt engineering, which relies on manually rewriting prompts, with prompt programming, a higher-level approach in which developers describe the task in code and let a framework generate and refine the prompts. DSPy is presented as the leading example of this approach, aiming to improve prompt quality while reducing development and maintenance effort.
DSPy’s key idea is to treat prompt creation like software development: define inputs, expected outputs, and evaluation criteria, then use automation to search for strong prompts. The chapter emphasizes the value of modular, testable, reusable code and explains that DSPy can systematically generate candidate prompts, evaluate them on data, and select the best-performing ones. This data-driven process is framed as a practical response to the limitations of manual prompt tuning, especially when prompts must be reliable, adaptable, and efficient across different language models.
The chapter also shows why this matters for real applications, including summarization, classification, RAG, chatbots, and agents, where many model calls and complex workflows make manual prompt tuning impractical. DSPy is positioned as especially useful for large or long-running systems, but also helpful in simpler cases because it can produce better prompts with little code. The recommended workflow is to build a baseline, evaluate it thoroughly, optimize prompts automatically, and then deploy, with the broader goal of making LLM application development more systematic, scalable, and maintainable.
Layers of code when working with DSPy. We usually need only work at the top level, which is the source code we create – using classes and functions provided by the DSPy layer below.
Prompt of optimization of a customer service intent classifier. The optimization process generated 3 candidate prompts that are each evaluated. In the end, the best is selected. In this case, that is the 2nd prompt, which has the highest score, 90%. DSPy also supports processes that modify and re-evaluate the prompts over several iterations.
An example control flow in which several LM calls are made. The specific set of LM calls executed and their specific content are determined by previous LM calls and calls to tools.
The three main stages of building an LM-based application with DSPy
Example creating a simple, baseline application in DSPy
Once we have a baseline application, or any other version of the application, we can evaluate this.
DSpy supports automatically optimizing the prompts used by an application.
Summary
- To get good results from an LM, it’s necessary to ensure that it’s given a good prompt.
- Manually creating and tuning prompts is often slow and ad-hoc. It also must be repeated for each LM that’s considered.
- Prompt programming, which supports automatically creating, evaluating, and optimizing prompts, provides a more modern alternative.
- DSPy is the state of the art in prompt programming. It allows for clean, understandable, and simple code that can be easily re-executed to test new LMs or new prompting techniques.
- Working with DSPy allows us to develop quickly, as much of the work interacting with LMs is now handled automatically by the framework.
- Using DSPy, it’s recommended to first create a baseline application, then evaluate it, then optimize it.
- Optimization works by having another LM suggest many candidate prompts and by carefully evaluating each of these.
- Optimization may execute over multiple iterations, and uses established optimization techniques to identify, during each iteration, progressively stronger candidate prompts.
- DSPy supports defining complex workflows.
- Where workflows contain multiple prompts, DSPy allows us to optimize the full set of prompts together, allowing us to create very effective applications.
Building LLM Applications with DSPy ebook for free
