Reporting Practice Matters: The Impact of Reference Choice on Chest X-ray Report Evaluation
Study shows radiology reporting-style variations in reference reports can flip rankings of chest X-ray report generation models; releases MIMIC-CXR-Ext-ReRef dataset.
The paper quantifies how variations in radiologists' reporting practices distort evaluation of radiology report generation (RRG) models, introducing a radiologist-informed taxonomy and the ReRef method for rewriting reference reports while preserving clinical meaning. On MIMIC-CXR with RadCliQ-v1, condensing normal-findings discussion caused Libra to drop from first to second while CheXOne rose from third to first among nine models. The authors release MIMIC-CXR-Ext-ReRef, a radiologist-validated dataset of 120 original/alternative reference pairs, arguing metrics conflate clinical correctness with stylistic conformity.
- Reference report style can materially change RRG model rankings
- ReRef rewrites references along a radiologist-informed taxonomy of practice variation
- Example ranking flip: Libra 1st to 2nd, CheXOne 3rd to 1st on MIMIC-CXR/RadCliQ-v1
- Releases 120 radiologist-validated reference pairs (MIMIC-CXR-Ext-ReRef)
- Products
- MIMIC-CXRMIMIC-CXR-Ext-ReRef
From the original article
1 min read at arxiv.orgRadiologists follow heterogeneous reporting practices. Two radiologists examining the same image and identifying the same clinical findings might nevertheless compose superficially distinct reports, varying in terminology, shorthand, formatting, and level of detail.
By Daniel P. Jeong · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.