BIABench: Evaluating AI agents on real-world bioimage analysis tasks
BIABench tests AI agents on 16 published bioimage tasks, where 3D and time-lapse work stays unsolved.
BIABench is an open benchmark of 16 bioimage-analysis tasks reconstructed from published studies, preserving their scientific questions, imaging data, and ground truth. The tasks span eleven subtasks and modalities from H&E histology to single-molecule localization microscopy. Each run gets an outcome score against field-standard metrics and a process score from a vision-language model using an expert rubric. Routine 2D tasks were solved well, but on some tasks with a third dimension or a time axis no agent scored above 0.19, and scores varied more across repeats than across agents.
- Sixteen tasks rebuilt from published bioimage studies with ground truth
- Scores cover outcomes and vision-language judgments of method quality
- No agent exceeded 0.19 on some 3D or time-lapse tasks
- Run-to-run variance exceeded differences between agents
From the original article
1 min read at huggingface.co
Artificial-intelligence (AI) agents hold promise for automating bioimage analysis, yet no benchmark evaluates whether they can carry out real-world analyses end to end. Such analyses are hard for agents because 2D images, 3D volumes and time-lapse sequences are often too large to read as context, so an agent must choose and run an analysis through code, specialized software and rendered views.
By Zixuan Pan, Davide Panzeri, Lukas Johanns · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.