BatchBench leaderboard tests AI robustness to biomedical batch effects
Original title: BatchBench: Benchmarking AI Robustness to Batch Effects and Domain Shift in Biomedical Images - A JUMP Cell Painting Leaderboard
The leaderboard uses JUMP-CP Cell Painting images from unseen experimental batches and data sources.
The paper introduces BatchBench, a leaderboard for how well AI systems hold up when biomedical images come from experimental batches and sources they were not trained on. It is built on JUMP-CP, described as one of the largest public biomedical imaging datasets and generated with the Cell Painting assay. The authors say JUMP-CP has strong batch effects and domain shift, and that domain-adaptation methods still leave large performance gaps under those shifts. No model scores or rankings are given in the abstract.
- BatchBench compares robustness on unseen batches and data sources.
- Underlying data is the public JUMP-CP Cell Painting collection.
- Authors say domain-adaptation methods still leave large gaps under shift.
- The abstract does not report leaderboard scores.
From the original article
1 min read at arxiv.orgDespite recent advances, current AI systems remain highly susceptible to domain shifts, particularly in biomedical imaging, where images are usually acquired under substantially different experimental conditions. Although recent domain adaptation methods have made substantial progress, significant performance gaps under domain shift persist.
By Sonja Topf · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.