GPT-6 Astra finds planted flaws in clinical models via queries
Original title: Auditing black-box clinical prediction models with a large language model through queries alone: a blinded test with planted flaws
Blind query-only audits caught the planted flaw in 69 of 72 runs, but 15% of checkable extra reports were absent from the model.
Researchers trained four clinical prediction model types on earlier International Stroke Trial patients, planted one flaw in each of 24 copies, and mixed them with eight clean models. GPT-6 Astra audited each model three times through a prediction-only tool, under rules fixed before any run. It found the planted flaw in 69 of 72 runs, while a purpose-written script found all 24. Many extra problem reports failed pre-specified checks, and only 11% of 601 unexpected relationships from clean models held in later-randomised patients.
Unconfirmed: A post hoc review judged most failed checks to be real but small steps.
- Models were logistic regression, an explainable boosting machine, gradient-boosted trees and a neural network.
- Surrogates had a median of 128 terms and mean R-squared of 0.976 on log-odds.
- Sixty percent of 1,089 checkable reports failed a check against average model behaviour.
- After a training-data check, 33% of unexpected relationships held, versus 30% for comparison.
- Organizations
- International Stroke Trial
- AI models
- GPT-6 Astra
From the original article
1 min read at arxiv.orgLarge language models (LLMs) are used to explain clinical prediction models, on the promise that they can say what a model computes, find its faults and even reveal new knowledge in it. We tested the three promises where the answers could be checked.
By Zhiyu Liu · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.