FrontierPhysics evaluates AI agents on end-to-end physics research
Original title: FrontierPhysics: Evaluating Agents for End-to-End Physics Research
FrontierPhysics tests 22 models across 56 physicist-authored tasks and finds no configuration publishes more than 46 of 168 rollouts.
FrontierPhysics requires agents to review literature, plan research, conduct experiments and produce manuscripts in nine physics domains. Combining deterministic verifiers with expert rubrics it assesses both correctness and publishability. Frontier agents show weaknesses in rigorous reasoning and expert intuition rather than physics knowledge gaps. No model exceeds 46 successful rollouts.
- 56 tasks from 42 physicists
- 9 domains across physics
- 168 rollouts evaluated
- Main weakness: reasoning and intuition
- Deterministic verifiers plus rubrics
From the original article
1 min read at arxiv.orgAI agents are playing an increasingly important role in physics research. However, existing physics benchmarks only evaluate fixed questions, isolated sub-tasks, or paper reproduction with given solutions, and rarely check research quality alongside correctness.
By Bingran You · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.