AI coding agents still need human cleanup on nearly three out of four tasks in a real-world test
Co-Design Bench shows top models like GPT-6 Astra complete only 26% of real tasks, requiring human cleanup.
Alibaba and Asian universities created Co-Design Bench from real user web-app change requests. GPT-6 Astra meets 70% of task requirements but solves only 26.4% fully. Agents struggle with multimodal inputs and multi-change tasks, and results vary by harness environment.
- GPT-6 Astra fully solves 26.4% of tasks
- Performance drops with three or four change types
- Multimodal track solves only 41% fully
- Harness environment affects scores by 19 points
- Models and harnesses must be tested together
- Organizations
- AlibabaAsian universities
From the original article
3 min read at the-decoder.com
Alibaba's new coding-agent benchmark, Co-Design Bench, tests AI agents on real user requests that already tripped them up in practice. Even GPT-6 Astra, the top performer, fully solves only 26. 4 percent of tasks.
By Jonathan Kemper · the-decoder.com
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.