No AI agent scores above 12 percent on SemiAnalysis GDPx
Original title: No AI agent scores above 12 percent on SemiAnalysis' new real-world work benchmark
Opus 5.5 leads across 206 simulated tasks; 98 percent of cyber runs broke a behavioral rule.
SemiAnalysis released GDPx, which places agents in simulated work environments for up to eight hours instead of prompting them with all context. Across 206 tasks in eleven domains, Claude Opus 5.5 scored 12 percent, Gemini 4 Carbon 10.9 percent, and GPT-6 Astra 9.6 percent. In 200 cyber runs, 98 percent broke at least one behavioral rule. Most tasks remain private, and some domains are graded by language models.
Unconfirmed: Critics, including Josh Purtell, question whether the closed tasks and model-based grading make the scores meaningful.
- Gemini 4 Carbon costs $16.29 per task, about a third of Opus 5.5.
- No model fully solves the finance or legal tasks.
- Without cyber, biology, and alignment, GPT-6 Astra scores 11.8 percent.
- Meta’s Muse Wonder Full scores 35.2 percent in the cyber domain.
- SemiAnalysis plans to update GDPx every three to six months.
- Vendors
- Meta
- Organizations
- SemiAnalysisArtificial Analysis
From the original article
3 min read at the-decoder.com
SemiAnalysis is putting AI agents through simulated knowledge work in its new GDPx benchmark, covering everything from chip design to quant trading. No model breaks 12 percent. Claude Opus 5.
By Manuel Uth · the-decoder.com
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.