OpenTumorBoard: A Real-World Benchmark of Multidisciplinary Tumor Board Discussion Trajectories
OpenTumorBoard benchmarks LLMs on 611 real multidisciplinary tumor-board cases and discussions.
OpenTumorBoard is a medical LLM benchmark with 611 patient cases and 19,157 discussion turns across ten specialist roles, transcribed from 12,534 minutes of public YouTube tumor-board recordings. It evaluates specialist answers to real questions and full board simulations that must reach consensus on therapy, surgery, next actions, and clinical-trial matching. Among 14 general and medical LLMs, the best scores are 3.43/5 for clinical equivalence and 2.78/5 for alignment with recorded conclusions. Supervised finetuning and reinforcement learning improve a held-out set, and three physicians rate case coverage, factuality, and consensus fidelity highly.
- Benchmark has 611 cases and 19,157 turns from public tumor-board videos.
- Best of 14 models scores 3.43/5 equivalence and 2.78/5 alignment.
- Supervised finetuning and reinforcement learning improve a held-out test set.
- Three physicians find high coverage, factuality, and consensus fidelity.
From the original article
1 min read at huggingface.co
Multidisciplinary tumor boards integrate multimodal clinical observations and longitudinal patient histories through specialist discussions, yet benchmarks rarely capture these real-world trajectories. We introduce OpenTumorBoard, a benchmark with 611 patient cases and 19,157 discussion turns across ten specialist roles, transcribed from 12,534 minutes of publicly available tumor board recordings on YouTube.
By Anqi Li, Zhixuan Ge, Yixuan Duan · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.