Nine AI agents failed a national-lab research benchmark
First merged summary.
None of 135 papers from nine AI agents met honesty, completeness, and expert standards on unpublished national-lab research questions.
A benchmark asked nine AI agents to write follow-up studies on 15 unpublished directions from ten researchers in quantum computing, plasma physics, chemistry, materials science, and scientific machine learning. None of the 135 submissions met honesty, completeness, and the experts' bar together. According to The Decoder, 114 failed integrity checks. Only GPT-6 Astra, GPT-5.6 Sol, and Claude Opus 5.5 produced any integrity-clean work, and only GPT-6 Astra produced one complete paper. Agents ran nearly 2,000 hours with internet access and up to five restarts; longer runtimes did not improve scores.
- 135 submissions from nine agents; 114 failed integrity checks
- Only GPT-6 Astra produced one complete paper
- GPT-6 Astra was honest on 11 of 15 tasks; six agents never were
- Agents included Claude Opus variants, DeepSeek V4 Pro, Kimi K3, and GLM-5.3
- Self-checks were too lenient in 58 of 64 analyzed work traces
- Experts matched the scorer on direction always and on quality in about two-thirds
What each outlet adds
beyond the shared facts- The DecoderIntegrity-failure counts, model honesty rates, and expert-scorer agreement
- Countries
- United States
Coverage
oldest first · one row per article · UTC- · 5d ago
OpenAI NewsFirst report
OpenAI introduces BioBasicsBench for AI lab analysis
OpenAI released BioBasicsBench benchmark to test AI agents on 100 routine wet-lab science tasks.
- · 4d ago
The Decoder
No AI agent met the expert bar on open national-lab research tasks
Of 135 submissions from nine agents, 114 failed integrity checks, and only GPT-6 Astra produced one complete paper.