Coding-agent harness design advances: ModularRSI recursive self-improvement and an empirical study of 176 harness configurations

Two new papers advance coding-agent harness design: ModularRSI evolves five harness modules via benchmark-disjoint recursive self-improvement, reporting gains on TB2.0 and SWE-Bench Verified, while an empirical study of 176 matched harness settings across…

Two related papers on agent harness design for coding tasks appeared this week. ModularRSI (Hugging Face daily papers, 2026-09-13) proposes a modular, generalizable recursive self-improvement (RSI) framework: it contrasts successful and failed trajectories for the same task and aggregates evidence across tasks to identify recurring behavioral deficiencies, then decomposes the evolvable harness into five modules — Agent Loop, Tool Use, Observation Management, Context Management, and Task Completion Detection — each evolved independently within a restricted scope and integrated with conflict resolution. Using 2,000 executable evolution tasks disjoint from evaluation benchmarks, it shows consistent gains on TB2.0 and SWE-Bench Verified and transfers across different foundation models. Separately, 'An Empirical Study of Harness Design for Coding Agents' (Hugging Face daily papers 2026-09-16; arXiv cs.AI/cs.LG/cs.CL 2026-09-17) holds the execution loop fixed while varying planning, action space, and context management, evaluating 176 matched configurations across four models on SWE-Bench Verified and Terminal-Bench 2.1. Its trajectory-level analysis finds context management chiefly prevents context-overflow failures and matters most under tight context-window budgets; rule-based elision before LLM summarization is the most efficient strategy, with recoverable elided content rarely helping; planning shifts from an accuracy scaffold for weaker models to a cost saver for stronger ones; and bash-capable models run effectively with bash-only tool interfaces at substantially lower cost. Note: the two papers use different Terminal-Bench versions in their evaluations — ModularRSI reports results on TB2.0, while the empirical study uses Terminal-Bench 2.1; both evaluate on SWE-Bench Verified.

• ModularRSI decomposes the agent harness into five independently evolving modules — Agent Loop, Tool Use, Observation Management, Context Management, and Task Completion Detection — with a conflict-resolving integration stage.
• ModularRSI identifies recurring behavioral deficiencies via contrastive analysis of successful vs. failed trajectories for the same task, aggregated across tasks.
• ModularRSI uses 2,000 executable evolution tasks disjoint from evaluation benchmarks to prevent benchmark-specific overfitting, reporting consistent gains on TB2.0 and SWE-Bench Verified and transfer across foundation models (Hugging Face…
• The empirical harness-design study evaluates 176 matched harness settings across four models on SWE-Bench Verified and Terminal-Bench 2.1, varying planning, action space, and context management with a fixed execution loop (HF daily papers…
• Context management mainly prevents context-overflow failures and matters most under tight context-window budgets; rule-based elision before LLM summarization gives the best efficiency among strategies, and recoverable elided content rarely…
• Planning helps weak-model accuracy but mostly saves cost for strong models.
• Bash-capable models perform well with bash-only tool interfaces at substantially lower cost.
• Sources describe different Terminal-Bench versions: ModularRSI reports on TB2.0, while the empirical study evaluates on Terminal-Bench 2.1; both papers also evaluate on SWE-Bench Verified.