OpenTumorBoard: A Real-World Benchmark of Multidisciplinary Tumor Board Discussion Trajectories
OpenTumorBoard tests LLMs on 611 real tumor-board cases; the best scores 3.43/5.
OpenTumorBoard contains 611 patient cases and 19,157 discussion turns across ten specialist roles, transcribed from 12,534 minutes of public YouTube tumor-board recordings. It evaluates specialist-turn replies and full board simulations covering therapy, surgery, next actions, and clinical-trial matching. Among 14 frontier and medical LLMs, the best scored 3.43 out of 5 for clinical equivalence and 2.78 out of 5 for alignment with recorded conclusions. Supervised finetuning and reinforcement learning improved a held-out set, and three physicians rated case coverage and factuality highly.
- 611 cases and 19,157 turns from public tumor-board recordings.
- Best models scored 3.43/5 equivalence and 2.78/5 conclusion alignment.
- Tasks cover therapy, surgery, next steps, and trial matching.
- Finetuning and reinforcement learning improved held-out results.
- Three physicians rated case coverage and factuality highly.
From the original article
1 min read at arxiv.orgMultidisciplinary tumor boards integrate multimodal clinical observations and longitudinal patient histories through specialist discussions, yet benchmarks rarely capture these real-world trajectories. We introduce OpenTumorBoard, a benchmark with 611 patient cases and 19,157 discussion turns across ten specialist roles, transcribed from 12,534 minutes of publicly available tumor board recordings on YouTube.
By Anqi Li · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.