Why Linear Models Can Still Work in a Complex and Changing World
Linear models can beat nonlinear ones when unstable curvature fails to transfer across environments.
The paper argues that linear models can outperform flexible nonlinear models when the world's complexity exceeds what finite, partially observed data can support. Nonlinear rules that fit one hospital, period, population, or measurement process may transfer weakly and raise squared error in a new environment. The claim is tied to bias-variance, regularization, best linear prediction, measurement error, distribution shift, invariance, Rashomon sets, and underspecification. Stable nonlinear structure still favors flexible models, and linear models remain exposed to linear spurious relationships, so complexity should be chosen by cross-environment stability as well as training accuracy.
- Nonlinear fits can raise error when they fail to transfer across environments.
- Finite-sample estimation error can strengthen the advantage of linear models.
- Partial observation matters only if links to hidden factors shift across settings.
- Authors recommend choosing complexity by cross-environment stability, not training accuracy alone.
From the original article
1 min read at arxiv.orgLinear models can predict well in complex domains even when underlying mechanisms are nonlinear. We argue that this can occur because the complexity of the world exceeds the complexity that can be learned reliably from finite, partially observed data.
By Fangzhou Lin · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.