2 Getting started improving agents

In this chapter, the focus is on making the simplest kind of self-improving agent work in practice: a retrieval-augmented generation agent whose system prompt is treated as the artifact to improve. The chapter starts by building a basic Helix-based agent and then attaching an archive so the prompt can be versioned, tracked, and recovered as the current best candidate. The result is a foundation for offline improvement, where the live agent stays stable while candidate prompts are tested and recorded.

The next major piece is the signal used to decide what counts as better. The chapter explains that evaluation can come from gold sets, deterministic checks, humans, or an LLM-as-Judge, and emphasizes the tradeoff between precision and availability. It then shows how to make the judge more reliable with a rubric and a swap-and-agree strategy that detects position bias by comparing answers in both orders and rejecting unstable judgments.

Finally, the chapter introduces search as the part of the loop that proposes better prompt variants, using Self-Supervised Prompt Optimization as the low-cost starting point. SPO mutates the current prompt, scores candidates with the judge, and keeps only improvements, creating a small hill-climbing loop that needs no labeled answer key. These components are then assembled into a complete offline improver in Helix, where the archive stores measured variants, the judge supplies the signal, and a human promotion step controls when the best prompt reaches production.

outlines the high-level components of the RAG agent that we will use in this chapter.

shows the offline improvement loop with an artifact archive which keeps a history of how each artifact was measured on the signal. Artifacts are ranked by score by calling best() and then promote() to advance the best artifact to the agent.

scatter plot demonstrating how precision and availability can guide your determination of how to evaluate good output from an agent

Outlines several ways in which a signal can be measured from what we establish is good or how to measure good.

details the internal process of how LLM-as-Judge uses pairwise evaluation to compare multiple outputs against the same rubric without having position bias modify the answer.

The shape every search method shares. Two inputs on the left, the current best artifact and the signal that scores variants, feed a search method that exposes the same two moves, propose and select. Whatever fills the slot (SPO here, GEPA and DGM in chapter 3), the output is identical: a selected variant that is a new artifact version with a parent pointer to its source. Because the shape never changes, the rest of the loop does not care which search ran.

The search methods on a cost-versus-reach map. Cost is what a method spends per run; reach is how high it can climb and how deceptive a landscape it can escape, and the methods fall on a rising frontier where more cost buys more reach. SPO is the cheap starting rung this chapter uses, with GEPA and DGM further up the ladder in chapter 3 and room marked for later methods. The gap between them only appears on a hard search, so on an easy landscape the cheaper method is the right call.

The search process. Search is the self-improvement loop run repeatedly: Propose generates a variant of the current best, Measure has the signal compare that variant against the current best, and Select applies the verdict, keeping the variant only when the signal preferred it. The signal does its work in Measure; Select is just the decision rule, with no second call to the signal. After the rounds run out, the best candidate becomes the new artifact.

How SPO works. SPO is the search loop of Figure 2.8 made specific: mutate is the propose step, score is the measure step, and accept is the select step. Each round mutates the current best prompt into a candidate, scores both prompts with the pairwise judge, and accepts the candidate only when it wins more than it loses, so the prompt never regresses. The self-supervised callout is the core idea: the only signal is which output wins the comparison, not a score against a labeled answer key, which is what makes SPO the cheap starting rung in Figure 2.7.

Putting it together with Helix. On the left, inside the OFFLINE boundary, the parts you instantiate (the Agent, the judge as the Signal, SPO as the Search, plus the EvalSource and Policy) compose into one OfflineImprover that runs the loop and writes every version and its score to the SQLiteArchive. On the right, the live agent serves the champion and is never touched by the loop. The two are bridged only by promote(), a human deploy gate; a winning candidate waits in the archive until someone opens it.

The OfflineImprover: the self-improvement loop, run offline. Demonstrating how the self-improvement loop is run offline. .

The ImproverPolicy. A single config object gates the loop along three axes: when rounds run (schedule), how much each round costs (questions, budget, per-question caps), and how a winner is chosen (promote_threshold_win_rate). The three teal fields are the only ones section 2.4 sets; the rest keep their defaults. Clearing the threshold marks a candidate promotable, but OFFLINE mode still waits for a human to call promote().

Section 2.4 in code. The seven steps of main_async, from attaching the console renderer to reading the final status. Steps two, four, and five are the substance, opening the archive, composing the four primitives, and wrapping them in one OfflineImprover; the rest is ordinary setup, run, and read-back.

• A RAG agent's system prompt is an artifact that an offline loop can improve without retraining the model.
• The examples favor cheap, fast, available models over frontier ones, leaving real headroom for the loop to close.
• The artifact archive stores every prompt version and its score, where best() ranks them and promote() ships the winner.
• A signal measures the gap between the agent's output and what good would have been.
• That standard can be a gold set, an LLM-as-Judge, a programmatic check, or a human, traded off along precision and availability.
• Every signal returns the same GapMeasurement, so the rest of the loop never cares which method produced it.
• An LLM-as-Judge is biased toward whichever answer it reads first.
• Swap-and-agree runs the comparison both ways and trusts the verdict only when the two runs agree.
• A rubric is the judge's system prompt, kept short, concrete, and ordered by importance so the order breaks ties.
• Search is generate-and-test: propose variants, measure each with the signal, and keep the winner as a new artifact version.
• Search methods sit on a cost-versus-reach frontier, so use the cheapest one that can reach your solution.
• SPO is self-supervised, never grading against a labeled answer, only asking which of two prompts wins and hill-climbing from there.
• Helix ships these as primitives that compose into one OfflineImprover, which clones the agent to test candidates and never mutates your original.
• Offline improvement only records what works, and promote() is the deliberate human deploy gate that ships a winner.

FAQ