Agent4Sci: frontier agents beat science ML baselines via engineering
Original title: Automating ML for Science: Can Frontier Agents Climb Scientific Hills in the Wild?
Claude Opus gained 10.4% mean relative improvement on 20 benchmarks; research plug-ins raised that to 20.1%.
Agent4Sci tests whether coding agents can improve published best models on 20 scientific ML benchmarks across 11 domains. Frontier agents advanced every benchmark, but almost entirely with ML-engineering tricks rather than domain insight. Plug-in skills for ideation, experiment management, and checklists lifted Claude Opus from 10.4% to 20.1% and Codex from 4.4% to 8.1%. No run was flagged for academic misconduct.
- The testbed spans 20 benchmarks across 11 scientific domains.
- Limits cited are novelty miscalibration, unused resources, and local sweeps.
- Codex mean relative improvement rose from 4.4% to 8.1% with plug-ins.
- Code and data are released on GitHub and Hugging Face.
- Products
- Agent4Sci
- AI models
- Claude OpusCodex
From the original article
1 min read at huggingface.co
Frontier coding agents are increasingly framed as end-to-end automated researchers, and ML for science offers a demanding test of that promise: improving a scientific model often requires domain insight beyond routine ML engineering.
By Ming Zhong, Stacy Li, Nicholas Carlini · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.