7 Custom modules
DSPy’s custom modules let you package prompting techniques and multi-step workflows into reusable building blocks that behave like built-in modules. The chapter explains that some modules are simple single-call wrappers that mainly add prompt content such as reasoning, constraints, or persona-like instructions, while others are workflows that coordinate multiple language model calls, logic, and even external tools. A key idea is composability: custom modules are built from sub-modules, usually Predict or ChainOfThought, so complex behavior can be assembled with relatively little new code.
The chapter then walks through how to code these modules in practice. A custom module is typically a Python class that subclasses dspy.Module and defines __init__() and forward(), with an optional aforward() for asynchronous use. __init__() usually creates sub-modules and adjusts signatures or instructions, while forward() handles preprocessing, calls to sub-modules, and postprocessing of outputs. Examples show modules that accept a fixed signature or a general signature via **kwargs, modules that add or rewrite prompt instructions, modules that filter toxic or unsafe input before answering, and modules that implement more advanced prompting styles such as concise reasoning or critic-driven revision.
Later sections focus on richer workflows and how to scale them. BestOfN, MultiChainComparison, Refine, and a critic-based module illustrate patterns for generating multiple candidate answers, comparing them, iteratively improving them, and looping until a quality threshold is met. The chapter also shows how to use asyncio to run model calls in parallel, both across modules and within a module, which can speed up systems that make many LM requests. Finally, it explains that custom modules can be optimized just like built-ins, including with BootstrapFewShot and MIPROv2, and even shows how DSPy itself can generate custom DSPy code and instructions for building more complex modules.
The workflow within the Refine module starts with the original prompt, which is used in a call to the LM. The response is collected, and Refine calls the reward method to evaluate the response. Each iteration, the LM is also called to get feedback about the prompt and the response, including advice on how to improve the prompt. In asking for this advice, Refine also provides the LM with task and reward function information, a history of the previous prompts, and how well the reward function scored their outputs. This advice is used as the hint field in the next call to the LM. Once the response is good enough (the reward function gives it a score above a specified threshold) or the process loops a specified number of times, the best response (as evaluated by the reward function) is returned.
Workflow used by a critic-based module. The three sub-modules—to execute a prompt, critique the response, and propose a new prompt—are executed in a loop until a response of sufficient quality, as judged by the critic sub-module, is found.
Summary
- Defining modules allows us to code prompting techniques and workflows.
- Creating custom modules usually requires relatively little code.
- Custom modules are Python classes that inherit from dspy.Module.
- They must define __init__() and forward() methods.
- Modules consist of one or more calls to sub-modules.
- Each module ultimately consists of one or more calls to Predict, which is the most basic module.
- Modules may also contain calls to tools and some logic.
- Once coded, custom modules can be used in the same way as built-in modules.
- Custom modules are also evaluated and optimized in the same way as built-in modules.
- In some cases, it may be useful to optimize specialized DSPy programs and pass these as parameters to other modules to perform some work, such as screening the input fields.
- Custom modules can themselves be used as sub-modules in other custom modules.
- Taking advantage of this, we can create modules with arbitrary levels of complexity.
- It’s possible to support asynchronous execution, which can make LM-based. applications substantially faster if there are LM calls that can be made in parallel.
- To support asynchronous execution, modules must provide an aforward() method.
- It’s possible to use DSPy to write custom modules.
FAQ
What are custom modules in DSPy?
Custom modules are user-defined DSPy classes that subclassdspy.Module. They let you package prompting techniques and multi-step workflows into reusable components, just like built-in modules such as Predict and ChainOfThought.What are the two main types of DSPy modules described in this chapter?
The chapter describes two broad types: modules that make a single LM call and encode a prompting technique, and workflow modules that make multiple LM calls, include logic, and may use tools. Workflow modules support more complex systems like agents and RAG pipelines.How do you define a basic custom module in DSPy?
Define a Python class that inherits fromdspy.Module, add an __init__() method to create sub-modules, and a forward() method to run them and return a dspy.Prediction. If you want parallel execution, you can also define aforward().Do custom modules always need a signature?
No. Unlike built-in modules, custom modules do not always need a full signature. If the module has a fixed purpose and fixed inputs/outputs, you can omit it. But custom modules can also accept arbitrary signatures when you want them to be general purpose.What is the role of sub-modules in a custom DSPy module?
Sub-modules are the internal DSPy components used by a module, oftenPredict or ChainOfThought. Custom modules are composed from these sub-modules, and the final logic usually orchestrates their calls and processes their outputs.How does ChainOfThought work as a built-in module?
ChainOfThought makes one LM call but modifies the signature to add a reasoning field. It prepends a rationale/output field to encourage step-by-step reasoning before producing the final answer.What does BestOfN do?
BestOfN runs a sub-module multiple times, scores each result with a reward function, and returns the best prediction. It can stop early if a response reaches a threshold score.Why would you use a custom module for tasks like toxicity checks or prompt injection detection?
Because you can separate safety checks from the main task and run them first. If a message is toxic or contains a prompt injection, the module can stop before the main LM call, and the safety checks can be optimized separately.What is Chain of Draft and why is it useful?
Chain of Draft is a variation of Chain of Thought that limits the reasoning text to short drafts, often around five words per step. It keeps the benefits of reasoning while reducing token usage, latency, and cost.Can custom DSPy modules be optimized and run in parallel?
Yes. Custom modules can be optimized with the same DSPy optimizers used for built-in modules, including approaches like BootstrapFewShot and MIPROv2. They can also support asynchronous execution withaforward() and can call sub-modules in parallel using asyncio.
Building LLM Applications with DSPy ebook for free
