Skip to content
NewA calmer, faster ZeroHour
StoryAI researchinfo

TwinLab-Bench finds lab-design agents still struggle; top score 45.0

2 sources · 2 articlesFirst reported Updated ()
What's new

First merged summary.

Merged summary · x-ai/grok-4.7 · rewritten as coverage arrivesSummaries by Vani AI

TwinLab-Bench finds frontier models still struggle on 52 real-lab tasks; Claude Fable 5.2 averages 45.0.

Hugging Face daily papers on November 18 described TwinLab-Bench, which tests AI agents that design their own experiments rather than follow a given workflow or choose predefined measurements. It covers 52 tasks in physical sciences, life sciences and engineering, with more than 100 experimental action types. A TwinLab agent executes designs against real laboratory records, and reports are scored against a hidden answer. The authors say frontier models still struggle and study where Claude Fable 5.2 fails. They report Fable 5.2 as the strongest model, averaging 45.0. Closed-book probes control for recalled answers.

Key facts
  • The benchmark has 52 tasks and more than 100 experimental action types.
  • Reports are scored against hidden answers from real laboratory records.
  • Claude Fable 5.2 is the strongest model at an average of 45.0.
  • Closed-book probes control for answers recalled from memory.
  • The TwinLab agent nearly eliminates readings that deviate from lab records.
Explain this AI news with Vani AIWhat was announced, what is new and why it matters, in a new chat

Timeline

dated events in the reporting · oldest first
  1. Nov 18, 2026Hugging Face lists the TwinLab-Bench AI-scientist paper. · Hugging Face daily papers

What each outlet adds

beyond the shared facts
  • Hugging Face daily papers52-task TwinLab-Bench; Fable 5.2 averages 45.0.

Coverage

oldest first · one row per article · UTC
  1. · 3d ago arXiv cs.AI / cs.LG / cs.CLFirst report

    SciResearch-Eval finds scientific agents overconfident when simulators omit physics

    On 93 tasks, most models' 95% intervals miss the truth, while GPT-6 Astra and Claude Fable 5.2 stay near calibrated.

  2. · 1d ago Hugging Face daily papers

    TwinLab-Bench tests AI scientists on self-designed laboratory experiments

    Across 52 physical, life-science and engineering tasks, the paper's strongest model, Claude Fable 5.2, averages 45.0.