Comparison of techniques for fine-tuning open-weight models for entity extraction from radiology reports
Fine-tuned Gemma-3-12B matches GPT-4o on hemorrhage extraction when distilled from real reports.
A 2x2 study fine-tunes Gemma-3-12B for multi-label intracranial hemorrhage acuity extraction from head-CT reports, crossing a classification head versus instruction fine-tuning and real GPT-4o labels versus synthetic reports. Distilled instruction tuning matched GPT-4o (macro-F1 0.845 vs 0.850) and beat the untuned base by 0.178 on 100 expert-adjudicated reports. Synthetic-data models never exceeded the base at any training size. Training and inference fit a single 24 GB consumer GPU.
- Distilled instruction-tuned Gemma-3-12B macro-F1 is 0.845 versus GPT-4o 0.850
- It exceeds the untuned open-weight base by 0.178 macro-F1
- Training-data source, not method, decided the result
- Synthetic reports never beat the untuned base
- Fine-tuning and inference fit one 24 GB GPU
- AI models
- Gemma-3-12BGPT-4o
From the original article
1 min read at arxiv.orgConverting free-text radiology reports into structured labels supports cohort building, quality assurance, and monitoring of clinical imaging models, but the strongest label extractors are hosted proprietary models whose use raises privacy, cost, and reproducibility concerns.
By Aawez Mansuri · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.