Agent S4 evolves interfaces and leads open-source OSWorld
Original title: Evolving Agent Interfaces
With Qwen3.8-27B it scores 21.3% on OSWorld V2, 3.8 times the same model with fixed interfaces.
A paper introduces Agent S4, a meta-agent that builds and revises computer-use interfaces from unlabeled task trajectories, then selects repositories by validation performance. On OSWorld V2, Qwen3.8-27B with the evolved repository reaches 21.3% binary score, above the paper's cited GPT-5.5 and Claude Opus 4.8 scores, at a fraction of their costs. On ScienceBoard, the same model rises from 41.4% to 58.6%. Code is open-sourced.
Unconfirmed: State-of-the-art and cost comparisons are the authors' reported figures, not independently confirmed here.
- OSWorld V2 comparisons cited are GPT-5.5 at 13.0% and Claude Opus 4.8 at 20.6%.
- The authors put Agent S4's cost at about 4.3% and 1.5% of those models.
- Evolution uses synthetic training tasks and unseen validation tasks.
- Code is at github.com/simular-ai/agent-s.
- Products
- Agent S4
- AI models
- Qwen3.8-27BGPT-5.5Claude Opus 4.8
From the original article
1 min read at arxiv.orgComputer-use agents are limited by the interfaces humans give them. An agent may type a spreadsheet cell by cell when a single line of code would do, or print all two thousand lines of a file through a shell to find one function. Both the interfaces and the knowledge of when to use each are built by hand, which cannot keep pace with new applications, workflows, and models.
By Saaket Agashe · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.