Skip to content
NewA calmer, faster ZeroHour

AHE-Beta evolves agent harnesses from user requirements

Original title: Agentic Harness Engineering Beta: Requirement-Centered Evolution of Agent Harnesses

arXiv cs.AI / cs.LG / cs.CLby Jiahang Lin ()1 min read2
AI summary · x-ai/grok-4.7Summaries by Vani AI

On held-out tasks, pass@1 rose from 21.9% to 33.3% while using 40.8% fewer tokens than Base Harness.

The paper introduces Agentic Harness Engineering Beta, a loop that collects simulated-user requirements, implements them as generalizable harness changes, and merges a change only after it resolves a verified requirement and passes review. On held-out tasks it reports higher pass@1 than Base Harness, Claude Code, Meta-Harness, and AHE, with lower token use than Base Harness. In a blind human test, users preferred AHE-Beta in 31 of 48 valid pairs. Without further evolution, gains transferred to GLM-5.3 and DeepSeek-V4.1-Flash.

Unconfirmed: Reported gains are the authors' benchmark and blind-test results, not an independent evaluation.

Key facts
  • Pass@1 rose from 21.9% to 33.3% versus Base Harness.
  • Token use was 40.8% lower than Base Harness.
  • Users preferred AHE-Beta in 31 of 48 valid pairs.
  • Transfer gains were 15.8% on GLM-5.3 and 5.3% on DeepSeek-V4.1-Flash.
Explain this AI news with Vani AIWhat was announced, what is new and why it matters, in a new chat

From the original article

1 min read at arxiv.org

Real-world harness engineering is driven by explicit and implicit user requirements, which guide development and validation. However, existing harness evolution methods often overlook implicit requirements and interaction needs, while using a single evolution agent to modify the harness and benchmark scores to select changes. This process often produces narrow patches and leads to benchmark-specific local optima.

arXiv cs.AI / cs.LG / cs.CL

By Jiahang Lin · arxiv.org

Read the full article at arXiv cs.AI / cs.LG / cs.CL

A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.