PhysClosureBench finds frontier agents often miss full physics research closure
Original title: From Research Goals to Scientific Closure: Evaluating Agents Through Evidence-Supported Physics Research
Claude-Opus-5 scores 6.29/10 on average but reaches full closure in only 32.7% of attempts.
Researchers introduce PhysClosureBench, 50 expert-verified physics research tasks that leave agents to organize evidence and validation. Seven frontier LLMs were scored with a 20-criterion Atomic Score and a Research Closure scale from RC0 to RC4. Claude-Opus-5 led on local accomplishment, yet many high-scoring attempts still failed to satisfy indispensable evidence and scope conditions.
- The benchmark covers strong-field, attosecond, and neighboring physics areas.
- 36.2% of attempts scoring at least 8/10 still missed RC4.
- All seven models reached RC4 on fewer tasks across all three runs than in at least one.
- Case analysis cites misread results and unresolved physical applicability limits.
- Products
- PhysClosureBench
- AI models
- Claude-Opus-5
From the original article
1 min read at huggingface.co
Can AI agents organize scientific investigations that lead to well-supported conclusions? To evaluate this capability, we introduce PhysClosureBench, a benchmark of 50 expert-verified, goal-specified research tasks spanning strong-field, attosecond, and neighboring physics areas.
By Yijia Mao, Song Dai, Tao Chen · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.