AI coding agents complete under one in ten workplace tasks
Original title: AI coding agents ace isolated tests but complete fewer than one in ten tasks in a simulated workplace
Claude Opus 5.5 and GPT-6 Astra near top SWE-bench scores but fully finish fewer than 10 percent of tasks as simulated employees.
Standard benchmarks such as SWE-bench are nearly saturated, and Anthropic says Claude writes more than 80 percent of its production code. A new experiment instead places coding agents in a simulated software company with colleagues, project boards, and chat history. Even Claude Opus 5.5 and GPT-6 Astra fully complete fewer than one in ten tasks. The snippet does not name the experiment or its scoring method.
Unconfirmed: The snippet does not identify the experiment or confirm how task completion was scored.
- Anthropic says Claude writes more than 80 percent of production code.
- Agents worked beside simulated colleagues, boards, and chat histories.
- The snippet does not name the lab behind the workplace test.
- Products
- SWE-bench
- Organizations
- Anthropic
- AI models
- Claude Opus 5.5GPT-6 Astra
From the original article
1 min read at the-decoder.com
A new real-world test puts AI coding agents in a simulated developer job with colleagues, project boards, and incomplete handoffs, where they have to keep developing software on their own and reliably close open tickets in a messy daily workflow. Even Opus 5. 5 and Astra fall far short, solving fewer than one in ten tasks completely.
By Jonathan Kemper · the-decoder.com
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.