2 Basic prompting and DSPy
Working with language models in software usually demands more reliable prompting than casual chat, because the prompt cannot be endlessly reworded until it happens to work. The text explains that effective prompts often contain familiar parts such as task description, context, input and output specification, output formatting, demonstrations, constraints, reasoning requests, emotional appeal, and a closing reinforcement. The chapter’s classification example shows how these pieces combine into a structured prompt designed to produce a predictable JSON response and handle ambiguity by returning a fallback label when needed.
The chapter then shifts from manual prompt engineering to DSPy, showing that a full application can be expressed with a small amount of code that creates the prompt, sends it to an LM, parses the response, and can even search for a better prompt variation. DSPy is presented as a higher-level way to work: instead of writing prompts directly, you define a model, configure it, provide examples, and call a program. It also explains practical setup concerns such as installing the library, managing API keys safely with environment variables or .env files, calling models directly, using LiteLLM for many providers, and understanding caching and model parameters like temperature and max tokens.
Finally, the chapter introduces DSPy’s core abstractions: signatures, modules, predictions, and the surrounding ideas of metrics, examples, evaluations, and optimizers. Signatures describe the input-output shape of a task in a concise way, while modules decide how to prompt the model, with Predict as the simplest module and ChainOfThought and ProgramOfThought representing more advanced strategies. Predictions are returned as structured Python objects whose fields match the signature. Overall, the chapter emphasizes that DSPy makes LM applications more readable, modular, and easier to review than large hand-written prompts, while still allowing the system to benefit from prompt optimization and flexible model swapping.
A structured diagram of a prompt. This flow includes specifying the response choices we want to receive.
Summary
- When working with LMs, either through prompt engineering or prompt programming, the goal is to create the best prompt we can for the task at hand and the LM used.
- Prompts are usually divided into multiple sections, chosen from a fairly consistent set of often-used sections.
- The ideal set of sections can vary from one LM to another and from one task to another.
- The specific content and format of these sections for the optimal prompt can also vary.
- DSPy provides a simple interface that requires specifying the LM, a simple signature, and a module to execute this.
- Any one of these components can be changed without needing to change the others; they are defined in a way that neatly separates the concerns.
- The signature may be provided in string format or as a Python class
- The module will convert the signature into a prompt, pass this to the LM, and return the response in the format specified in the signature.
- The module controls the prompting technique, which determines, among other things, the sections included in the prompt.
- Each instance of a module we create is referred to as a program.
- The program is called as a function, passing the input fields specified in the signature.
- It returns an object of type Prediction.
- To optimize the prompt, we don't require many additional lines of code.
FAQ
What is the main difference between prompt engineering and DSPy?
Prompt engineering manually tweaks prompt wording, order, and sections through trial and error. DSPy instead lets you specify the task at a higher level using signatures and modules, and it generates and can optimize the prompt for you.What are the common sections of an effective LM prompt?
Common prompt sections include task description, context, input/output description, output format description, demonstrations, constraints, chain-of-thought requests, emotional appeals, and a closer. Not every prompt needs all of them, but these sections often improve reliability.Why are long prompts often necessary in LM-based applications?
When software makes LM calls, the output has to be reliable, so prompts are often more detailed than casual user prompts. Longer prompts help provide clearer instructions, examples, format requirements, and constraints.What is a signature in DSPy?
A signature is a concise specification of the inputs and outputs for a DSPy program, such asquestion -> answer or message, labels -> intent_label. It describes what the LM receives and what it should return.What is the role of a DSPy module?
A module specifies how the task should be prompted. For example,Predict, ChainOfThought, and ProgramOfThought are modules that correspond to different prompting techniques.What does a DSPy prediction object contain?
A prediction is a Python object containing the output fields defined by the signature. You can access its values with dot notation likeprediction.answer or with square brackets like prediction['answer'].How do you set up and configure an LM in DSPy?
You create an LM object withdspy.LM(...) and then pass it to dspy.configure(lm=lm). After that, DSPy uses that model for subsequent calls until you switch models.How can API keys be stored securely for DSPy?
The chapter recommends storing API keys in environment variables, often loaded from a.env file using python-dotenv. This keeps secrets out of source code and helps avoid accidentally committing them to version control.
Building LLM Applications with DSPy ebook for free
