RRSI: Regularized Recursive Self-Improvement of Agent Harnesses
Google Research's RRSI regularizes recursive agent-harness edits, lifting held-out scores while cutting policy tokens.
Google Research introduces Regularized Recursive Self-Improvement (RRSI) to constrain how LLM agent harnesses evolve their prompts, tooling, memory, and control flow. An annealed edit budget and history-based exploration limit candidate changes, while a critic and pruner reject benchmark-specific, tiny, expensive, or obsolete edits. Across eight benchmarks, RRSI gains up to 14.1 points in distribution and 4.7 on five out-of-distribution tests, using 30% fewer policy tokens than unregularized evolution.
- RRSI anneals edit budgets and favors unexplored harness-evolution trajectories.
- A critic screens benchmark-specific proposals and a pruner drops weak or costly changes.
- Gains reach 14.1 in-distribution points and 4.7 on five held-out benchmarks.
- The resulting harness uses 30% fewer policy tokens than unregularized evolution.
- Evaluated across eight coding, workspace, and engineering-design benchmarks.
Full article229 words · extracted from arxiv.org · click to collapse
An LLM agent's capability is largely magnified by its harness, namely the prompts, control flow, tooling, memory, and context management surrounding the frozen backbone model. Recent methods increasingly automate this process by iteratively proposing and selecting component-wise edits of an agent harness, practically establishing a form of recursive self-improvement (RSI) at the agent-system level. However, such recursive evolution may overfit by memorizing the training tasks, showing large in-distribution gains that shrink or even vanish on out-of-distribution benchmarks. We introduce Regularized Recursive Self-Improvement of Agent Harnesses (RRSI), which incorporates the principles of regularizations into harness self-improvement by constraining the evolution candidate proposal and selection. The proposer operates with a temporally annealed budget, limiting how many edits a candidate can bundle, and it encourages unexplored trajectories based on evolution history. The selector is equipped with a critic and a pruner: the critic screens benchmark-specific proposals, while the pruner, removes changes that are too small, too expensive, or no longer useful. Together these constraints favor reusable agent mechanisms over benchmark-specific ones or even noises. Across eight benchmarks spanning coding, agentic workspace and engineering design tasks, RRSI gains up to 14.1 points on the split it evolves against and up to 4.7 points on the five out-of-distribution benchmarks, while producing a harness that runs on 30% fewer policy tokens than the unregularized evolution. Code is available at https://github.com/google-research/rrsi and project page is https://regularized-rsi.com/.
Text extracted automatically; images, tables and formatting may be missing. Original: https://arxiv.org/abs/2609.24972