Learn more：https://treerouter.com

Abstract

The Harness² framework introduces a two-layer recursive mechanism for continuous harness optimization while model weights remain frozen. Task-level recursion optimizes the active harness for the ongoing task through a Propose–Probe–Compose loop. Evolution recursion transfers validated harness edits and execution traces into an experience library, supporting cross-task migration. Its core editable surface is split into eight modular component types. Agent capability is determined not only by the base large model, but also by the surrounding harness: system prompts, memory banks, skill definitions, tool bindings, and task scaffolding. This relationship is commonly formalized as Agent = Model + Harness. Northwestern University and Google DeepMind researchers first outlined this design in the paper *Harness²: Recursive Agent Harnessing for an Open World*.

Open-ended agent workflows create two hard constraints for static harnesses. First, novel tasks often lack ground-truth answers or reliable verifiers, making it difficult to confirm whether a harness modification delivers functional gains. Second, harnesses tuned tightly for one task tend to overfit to domain-specific patterns, and fail to generalize cleanly to subsequent unseen tasks. Harness² addresses this by separating the optimization pipeline into two recursive loops. Task recursion revises the harness to fit the current task; evolution recursion packages actionable lessons from that tuning cycle for reuse on future workloads. This article breaks down its component design, recursive execution flow, benchmark results, failure modes, and operational tradeoffs.

1. Eight Editable Component Groups of the Harness

The authors decompose the adjustable harness surface into eight discrete component categories. Each maps to a distinct runtime artifact, so every harness revision can be tracked as file-level diffs.

  • System prompts, constraint rules, long-term memory, and tool instructions are reloaded on every model invocation.
  • Skill definitions are loaded lazily on demand.
  • Scripts run arbitrary operations via Bash shells.
  • Sub-agents handle delegated subtasks within isolated execution contexts.
  • Plugins execute inside the tool-use loop, injecting custom logic around API calls and tool iteration.

Harness modifications are not limited to prompt text alone. Flow logic can be encoded into skills, deterministic operational steps can be captured as scripts, independent sub-context workloads can be offloaded to child agents, and runtime guardrails can be enforced through plugins. The two recursive layers operate over this full set of editable artifacts, proposing, testing, and merging component changes iteratively.

2. The Two Recursive Layers of Harness²

The architecture divides work into Task-Level Recursion and Harnessing-Level Recursion (Evolution Recursion), as visualized in Figure 1.

2.1 Task-Level Recursion: Propose–Probe–Compose

The atomic cycle for task recursion follows three steps: propose edits, probe candidate harness variants, and compose the final updated harness.

  1. Propose: The Harness Editor generates candidate harness revisions from two directions. Generic candidates refine universal rules, tool-use protocols, and task delegation patterns. Domain candidates inject niche workflow constraints and domain knowledge.
  2. Probe: Each candidate harness runs the identical target task under the same base model and execution sandbox. The system logs full traces and output artifacts. It then compares runtime behavior between the baseline and candidate harness as a behavioral control. Behavioral contrast provides evidentiary support for edits, but cannot conclusively prove task correctness, because different valid implementations can produce divergent traces, and model variance introduces randomness.
  3. Compose: The Harness Composer evaluates all candidate traces, file diffs, and execution metrics. It merges selected changes, adjusts conflicting rules, or retains the original harness unchanged. The merged artifact becomes the active harness for the next iteration of the task.

A concrete example from the JobBench benchmark illustrates this pipeline. The baseline harness scored 43.1. Trace inspection revealed two critical gaps: the agent did not cross-check numeric values across separate source files, and it omitted post-write validation after output generation. The editor added generic input sanitization and verification scripts, paired with domain-specific reconciliation skills and targeted numeric validation tool guidance. After composition, the revised harness reached a final score of 77.8, a gain of +34.7 points.

2.2 Parallel and Sequential Recursion for Wider Exploration

After completing one Propose–Probe–Compose round, Harness² can scale exploration with two recursive variants: parallel recursion and sequential recursion.

  • Parallel recursion: Multiple independent harness candidates run simultaneously. Each candidate explores distinct domain invariants. The composer aggregates results from all parallel branches to build one consolidated final harness. This expands the search breadth within a fixed time window.
  • Sequential recursion: Candidates run in ordered rounds. Each new round inherits the harness state and accumulated proof records from the prior run. The system can retain, tweak, or roll back edits from earlier iterations.

Both modes preserve execution traces and harness snapshots, not just final outputs. When k>1, a selector picks validated harness variants from finished runs to feed into composition; candidates that do not produce distinguishable improvement are discarded and not written into the experience library. The team tested values of k from 1 to 5 on WB-Code and WB-Office benchmarks. Performance generally rises with higher k within this tested range, though results on Harness Scaling and ReasoningBank show unstable returns. Increased recursion budget does not guarantee linear performance uplift.

3. Evolution Recursion: Writing Validated Edits into the Experience Library

Once a task completes, the Reflector reviews the entire tuning history: all candidate harnesses, probe traces, composition decisions, and end outcomes. It classifies each modification with evidence labels: helpful, harmful, inconclusive, or unattributable. The unattributable tag applies when multiple component edits are bundled in one candidate, and execution results cannot isolate which individual change drove the outcome. The framework avoids pushing unproven causal claims into long-term memory.

Validated changes enter the experience library as persistent records. Each entry stores usage context, transferable edit patterns, supporting execution traces, and a confidence score. If no reliable behavioral contrast is established for a candidate, it is marked NO_ENTRY and discarded. The library stores full adopted harness snapshots and a central EXPERIENCE.md index, so future tasks can retrieve complete records, or selectively reuse and adapt individual skills, scripts, and rule files.

Crucially, every new task starts from an initial blank harness state H0. It does not directly inherit the final harness from the prior job. Instead, the editor loads the experience library at the Propose, Probe, and Compose phases to reuse historical insights. Base model parameters remain frozen throughout the whole sequence; only harness scaffolding evolves across tasks.

4. Quantitative Benchmark Results for Cross-Task Transfer

The researchers first measured single-task improvement with k=1. Across three benchmarks and four model + harness starting configurations, Harness² consistently outperformed static baseline harnesses. The largest gains were +11.5 points on WorkBuddy-Bench, +4.5 points on JobBench, and +11.8 percentage points on the LAB pass-rate metric.

Ablation studies isolated the contribution of the experience library, editing scopes, and composition steps. Under the Gemini 3.5 Flash + OpenCode setup with k=1:

  • Full Harness² scored 77.5 on WorkBuddy-Bench and 36.0 on JobBench
  • Removing the experience library dropped scores to 74.4 and 34.2
  • Removing both generic and domain editing scopes fell to 74.6 and 32.3
  • Removing the composition step fell to 75.9 and 34.1

A separate ablation tested direct harness editing without intermediate probe executions. When the editor reused library records without fresh execution traces, scores degraded sharply: WB-Code fell from 73.4 to 63.2, and WB-Web fell from 69.9 to 65.0. With k=1 library reuse, performance partially recovered close to baseline harness levels. With k=5 accumulated experience, WB-Code, WB-Web, and WB-Office improved by +0.9, +4.4, and +5.1 points relative to the static baseline. This confirms the second-layer recursion value: running extra harness exploration on one task generates traces and composition decisions that generalize and improve harness edits on future unrelated workloads.

Cross-benchmark transfer tests further validated this property. When JobBench experience records were injected into an existing WorkBuddy-Bench harness, WB-Code, WB-Web, and WB-Office improved by +0.6, +0.5, and +2.9 points respectively. Transfer performance varies widely across base models and benchmark domains. The authors ran tests on OpenCode with k=2, letting the same model handle both task execution and harness editing. Five tested models ranging from GLM-5 to Claude Opus 5 all exceeded their static baseline harness scores, proving the recursive tuning pipeline does not depend on a separate, stronger specialized editor model.

Component reuse statistics show a nuanced pattern. Only 2% to 10% of revised files retain half or more of their source code from a matching library entry. Harness adaptation is incremental refinement rather than verbatim copy-paste. Combined, skill and script artifacts account for roughly half of all component edits across the benchmark suite.

Teams running repeated model benchmarking campaigns often need unified access to diverse model endpoints and centralized usage logging. A managed API gateway such as Treerouter can simplify multi-model evaluation workflows, consolidating routing and quota tracking for large batches of harness variants. This operational convenience becomes meaningful when running many parallel probe cycles under controlled sandbox conditions.

5. Limitations and Failure Modes of Behavioral Contrast

Harness² relies on behavioral contrast to judge edit quality, and this signal carries measurable error. The paper catalogs nine distinct failure archetypes:

  1. Blind selection of weaker candidates: The composer picks a candidate harness that looks favorable on partial traces but degrades overall end-task performance. In one WorkBuddy-Code case, the selector chose a harness scoring 55, while an alternate candidate reached 95; the selected edit introduced a -41 point gap.
  2. Edit pool saturation: New valid edits are blocked by prior rule constraints baked into the harness. The search space shrinks prematurely even when better solutions exist.
  3. Misattribution from correlated traces: Shared execution noise or model bias creates false positive evidence. The system attributes a score jump to a harness change when the gain actually came from stochastic model sampling.
  4. Unreviewed result volatility: Single noisy successful runs are treated as proof of a stable improvement, without repeated validation.

The authors explicitly note that behavioral contrast cannot replace ground truth verification. Without canonical answers, trace comparison is vulnerable to sampling variance and shared model bias. A harness validated for one task may not transfer reliably to other domains. Even edits that produce consistent local gains can trigger cascading regressions on downstream unseen subtasks.

6. Computational and Operational Overhead

Every recursive tuning cycle adds compute cost. Each candidate harness spawns full inference runs, tool calls, and trace logging. Parallel recursion cuts wall-clock latency but does not reduce total token consumption. The paper characterizes performance and compute tradeoffs only within its tested budget range; it does not propose a universal cost threshold for all agent workloads.

The experience library itself requires maintenance overhead. As records accumulate, teams need mechanisms for deduplication, stale entry retirement, semantic search indexing, and confidence decay. Optimal library management will change dynamically based on task difficulty, domain similarity, and the volume of incoming harness traces.

7. Conclusion

Harness² demonstrates a practical recipe for agent improvement under frozen base weights. Task-level recursion iteratively shapes the harness for the active task using the Propose–Probe–Compose loop. Evolution recursion preserves the evidentiary traces of that tuning work into a reusable experience library, enabling cross-task transfer of validated scaffolding changes. Benchmark results confirm measurable gains on WB-Code, WB-Web, WB-Office, JobBench and WorkBuddy-Bench, with the largest benefits appearing when the system combines library reuse with fresh probe executions.

The core insight is that agent improvement does not always require fine-tuning model weights. Many failures stem from weak scaffolding: incomplete validation rules, missing domain skills, poor tool-use instructions, or unsafe plugin guardrails. The framework’s biggest caveat is that behavioral contrast is an imperfect supervision signal. It can produce false positives, misattribute gains, and select regressive harness variants. For production deployment, human review and occasional ground-truth validation remain necessary guardrails alongside automated recursion.

This two-layer recursive design provides a blueprint for building self-improving agent systems that learn to tune their own operating environment, rather than just learning the task itself.