Incidental information contaminates patient notes and disrupts clinical reasoning in large language models
Frontier LLMs insert small talk into 35% of clinical notes and sometimes treat it as clinical fact.
Across 576 patient-clinician dialogues, frontier models inserted small-talk exchanges into 35% of clinical notes, while mean quality scores changed by at most 0.20 on five-point scales. In 3.7% of those notes, models misattributed asides or used them clinically. In 57 mock consultations, background speech from another encounter at -10 dB leaked into 48.2% of transcripts and appeared in 5.3% of notes from four open-weight models. The authors argue that components linked to this distraction also support clinical reasoning and call for pre-deployment resistance tests.
- Small talk entered 35% of frontier-model clinical notes.
- 3.7% of frontier notes used incidental remarks clinically.
- Background speech contaminated 48.2% of transcripts and 5.3% of notes.
- Authors recommend testing incidental-information resistance before clinical use.
From the original article
1 min read at huggingface.co
Large language models (LLMs) are increasingly relied upon to support ambient documentation and clinical reasoning. Here we examine the impact of a failure mode shared between these two applications by assessing their sensitivity to information incidental to the patient encounter.
By Krithik Vishwanath, Brandon Ye, Anton Alyakin · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.