5 Resampler: Reliability through repeated sampling

When a generative model is asked the same question more than once, it may produce different answers because its behavior is probabilistic rather than fully deterministic. In everyday chatbot use, that variability is often just annoying, but in agentic software it can become a serious reliability problem when orchestration code expects a response in a precise format. If the model drifts from the requested structure, loops in its reasoning, or fails to finish within budget, downstream code may not be able to recover from the malformed output.

The core remedy is to wrap the model in a loop that keeps sampling until the output satisfies a chosen criterion. This pattern combines a worker element that produces a candidate result with an assessor that checks whether the result is acceptable, then either returns the result or tries again. The design improves reliability when used to enforce formatting, quality, or safety constraints, while also making stochasticity useful for exploration, such as searching for alternative solution paths or generating varied but valid outputs. The tradeoff is higher cost from repeated trials, and the approach works best when failures are caused by chance rather than by a deeper systematic flaw.

The chapter gives several concrete applications. A reasoning component can be resampled until it produces a final answer in the required markup, a summarizer can be resampled until it yields a brief neutral summary, and a chemistry-focused agent can be paired with random input subsampling to discover different reaction chains leading to a target compound. It also introduces a variant where a guardrail is combined with resampling so that only compliant outputs are released, and notes that input randomization can help when the worker itself is not stochastic enough. Overall, the pattern is presented as a practical way to turn the unreliability of AI systems into either stronger constraint enforcement or broader creative search.

AFD of the Resampler pattern. The input passes through the worker IE, which produces output. An element that may or may not be intelligent then assesses whether the output satisfies a prespecified criterion. Depending on the result, either the output is forwarded as the resampler’s final output or control returns to the worker IE for a new trial based on the same input.

AFD of Resampling Guardrail. The input first passes through the worker agent. The worker agent produces output, which the guardrail then assesses. Depending on the guardrailing assessment, either the output is forwarded as the final output or control is handed back to the worker agent for another trial.

Generic AFD of an input-stochastic resampler. Compared with figure 5.1, note the random augmentation step at the beginning of every trial. Random augmentations can be intelligent (e.g., rephrasing language input) or non-intelligent (e.g., sampling a random subset of input documents or masking random patches of an image).

AFD of a reasoning reliability resampler. The reasoner agent is the worker IE in this instance. The assessment element is simply a code procedure that tests for the presence of a pattern expressed as a regular expression. Both the final answer and the reasoner’s entire output can be provided as the resampler’s output.

AFD of the style enforcement resampler. The style conformity assessment is based on both the input text and the summary to better judge the summary’s neutrality. The element assessing the summarizer output for style conformity is intelligent.

AFD of a stochasticity-using resampler. Note the random but nonintelligent sampling of reactant subsets that precedes the invocation of the chemistry specialist.

• The randomness of generative AI models can cause agents to fail when strict output formats are required, and retrying the task can often recover from these errors.
• The Resampler pattern wraps an intelligent element in a loop that repeats until its output satisfies a criterion.
• Resamplers can suppress or exploit stochasticity, constraining randomness to improve reliability or harnessing it to explore alternative solutions.
• Resamplers depend on repeated independent trials without feedback, making them simple yet effective for enforcing quality, safety, or style requirements.
• Practical variants, such as Resampling Guardrails and input-stochastic resamplers, help balance reliability, cost, and creativity.
• Practical examples of Resampler in action, including reasoning reliability checks, stylistic enforcement, and reaction-chain discovery, show Resampler to be a versatile design pattern for mitigating unreliability and turning stochasticity into a creative asset.

FAQ