Prospective Prediction of OOD Degradation from Source-Side Training Dynamics
Source-side training dynamics can prospectively predict later out-of-distribution model failure.
The paper asks whether persistent out-of-distribution degradation can be predicted before it is observed, using only source-side training dynamics. In a controlled shortcut-learning setting, logistic regression finds a prospective signal that training time alone does not, and temporal summaries outperform current values. Confidence and entropy dynamics transfer from a CNN to an MLP without extra training, offering an early-warning proof of principle for future OOD failure.
- Predicts persistent OOD degradation before it is observed
- Logistic regression finds a signal training time alone lacks
- Temporal summaries beat current source-side values
- CNN confidence and entropy dynamics transfer to an MLP
From the original article
1 min read at arxiv.orgWe study whether persistent out-of-distribution (OOD) degradation can be predicted before it is directly observed using only source-side training dynamics. In a controlled shortcut-learning setting, a simple logistic regression predictor develops a clear prospective signal, while training time alone does not.
By Sasha · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.