TwinLab-Bench finds lab-design agents still struggle; top score 45.0
First merged summary.
TwinLab-Bench finds frontier models still struggle on 52 real-lab tasks; Claude Fable 5.2 averages 45.0.
Hugging Face daily papers on November 18 described TwinLab-Bench, which tests AI agents that design their own experiments rather than follow a given workflow or choose predefined measurements. It covers 52 tasks in physical sciences, life sciences and engineering, with more than 100 experimental action types. A TwinLab agent executes designs against real laboratory records, and reports are scored against a hidden answer. The authors say frontier models still struggle and study where Claude Fable 5.2 fails. They report Fable 5.2 as the strongest model, averaging 45.0. Closed-book probes control for recalled answers.
- The benchmark has 52 tasks and more than 100 experimental action types.
- Reports are scored against hidden answers from real laboratory records.
- Claude Fable 5.2 is the strongest model at an average of 45.0.
- Closed-book probes control for answers recalled from memory.
- The TwinLab agent nearly eliminates readings that deviate from lab records.
Timeline
dated events in the reporting · oldest first- Nov 18, 2026Hugging Face lists the TwinLab-Bench AI-scientist paper. · Hugging Face daily papers
What each outlet adds
beyond the shared facts- Hugging Face daily papers52-task TwinLab-Bench; Fable 5.2 averages 45.0.
- Products
- TwinLab-Bench
Coverage
oldest first · one row per article · UTCSciResearch-Eval finds scientific agents overconfident when simulators omit physics
On 93 tasks, most models' 95% intervals miss the truth, while GPT-6 Astra and Claude Fable 5.2 stay near calibrated.
TwinLab-Bench tests AI scientists on self-designed laboratory experiments
Across 52 physical, life-science and engineering tasks, the paper's strongest model, Claude Fable 5.2, averages 45.0.