BRIDGE gate blocks false model-update gains in retrospective tests
Original title: BRIDGE: An Acceptance Gate to Block False Improvements in Model Updates, with a Retrospective Clinical Evaluation
False-improvement rates fell from about 12–26% under a naive rule to 0–1.7% across five dataset groups.
BRIDGE is a model-agnostic acceptance gate that accepts an update only when the lower bootstrap bound of its gain exceeds a fixed margin, using cross-fitting so selection and judgment use different folds. On 11 new datasets, synthetic tasks, 12 retrospective clinical tasks, 39 supporting datasets, and 14 OpenML datasets, false-improvement rates dropped from 19.1%, 20.7%, 20.0%, 12.5%, and 25.6% to 1.1%, 1.7%, 0%, 0.5%, and 1.0%. The authors report BRIDGE removes 97% to 100% of false improvements the naive rule accepts, with 1.8 to 3.1 times the mean gain per deployed update.
Unconfirmed: Confidence intervals for the reduction are not taken over the clinical sources.
- Cross-fitting lets each fold judge an update chosen on the other folds.
- Every 95% interval for the reduction over datasets or tasks excludes zero.
- A 95% upper bound on the false-improvement rate is below 5.5%.
- The formal bound is 3.56% for independent synthetic tasks in setting B.
- Equal-size draws from naive acceptances held more false improvements (p<0.001).
From the original article
1 min read at arxiv.orgModels are updated repeatedly, and an update that looks better on validation data but not on new data is a false improvement that can misguide clinical decisions. The naive rule accepts every positive validation gain and is optimistic when the same data choose and judge the update. BRIDGE is a model-agnostic acceptance gate that decides each update from predictions and the available labels, with fixed settings.
By Dwi Sunaryono · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.