Skip to content
NewA calmer, faster ZeroHour

AI coding agents complete under one in ten workplace tasks

Original title: AI coding agents ace isolated tests but complete fewer than one in ten tasks in a simulated workplace

The Decoderby Jonathan Kemper ()1 min read
AI summary · x-ai/grok-4.7Summaries by Vani AI

Claude Opus 5.5 and GPT-6 Astra near top SWE-bench scores but fully finish fewer than 10 percent of tasks as simulated employees.

Standard benchmarks such as SWE-bench are nearly saturated, and Anthropic says Claude writes more than 80 percent of its production code. A new experiment instead places coding agents in a simulated software company with colleagues, project boards, and chat history. Even Claude Opus 5.5 and GPT-6 Astra fully complete fewer than one in ten tasks. The snippet does not name the experiment or its scoring method.

Unconfirmed: The snippet does not identify the experiment or confirm how task completion was scored.

Key facts
  • Anthropic says Claude writes more than 80 percent of production code.
  • Agents worked beside simulated colleagues, boards, and chat histories.
  • The snippet does not name the lab behind the workplace test.
Explain this AI news with Vani AIWhat was announced, what is new and why it matters, in a new chat

From the original article

1 min read at the-decoder.com

A new real-world test puts AI coding agents in a simulated developer job with colleagues, project boards, and incomplete handoffs, where they have to keep developing software on their own and reliably close open tickets in a messy daily workflow. Even Opus 5. 5 and Astra fall far short, solving fewer than one in ten tasks completely.

The Decoder

By Jonathan Kemper · the-decoder.com

Read the full article at The Decoder

A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.