Decision models extract cancer pathology fields with mixed accuracy
Original title: Can decision models extract clinical data from cancer pathology reports? An evaluation of accuracy, calibration, and failure modes
Tev1 4B matched local Qwen3.5 9B near 0.92 mean accuracy and finished each report about four times faster.
Researchers tested open-weights decision models Nimble 9B and Tev1 4B on extracting 25 fields from 1,113 oral cavity cancer pathology reports from one centre, against a clinical database. Generative baselines were local Qwen3.5 9B and cloud GLM-5.3-flash. Tev1's mean field accuracy was 0.917, statistically similar to Qwen3.5 9B, while Nimble reached 0.862 and GLM-5.3-flash 0.986. Weak spots included binned measurements, unreported answers, and some complex decisions. Escalating Tev1 answers below probability 0.94 to GLM-5.3-flash raised accuracy to 0.982, with 38% of answers sent on.
Unconfirmed: Results come from one centre and a simulated escalation strategy, not a deployed clinical workflow.
- Tev1 took 15 seconds per report versus 61 seconds for Qwen3.5 9B.
- Nimble improved when limited to two questions per request instead of 25.
- Pooled expected calibration error was 0.022 and 0.026.
- Returned probabilities did not flag systematic errors within single fields.
- Both decision models matched GLM-5.3-flash on simple dichotomous categories.
From the original article
1 min read at arxiv.orgDecision models are a new class of language model that, instead of generating text, choose one option per question from a fixed list and return a probability for every option. They have not yet been evaluated on clinical text.
By Indranil Mallick · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.