Revisiting Tabular Deep Regression: Objectives and the Limits of Global Early Stopping
Tabular deep regression study finds quantile loss beats tuned MSE, but global early stopping still compromises noisy regions.
The paper re-evaluates regression objectives for tabular deep learning across established datasets and multiple neural backbones. Many alternatives offer limited or inconsistent gains over a properly tuned mean squared error baseline, while quantile loss improves aggregate performance consistently. The authors show that regions with different target noise can peak at different epochs, so one global early-stopping checkpoint is a compromise. They propose a heteroskedasticity-aware quantile loss using input-dependent label smoothing, which improves several backbones.
- Many proposed regression objectives beat a tuned MSE baseline only inconsistently.
- Quantile loss gives consistent aggregate gains across tested neural backbones.
- Global early stopping can compromise regions with different target noise.
- Input-dependent label smoothing improves quantile loss on several backbones.
- Organizations
- Yandex Research
From the original article
1 min read at arxiv.orgWhile deep learning models for tabular data have recently received significant research attention, the choice of objective functions for regression tasks remains an understudied degree of freedom. Several lines of research offer alternatives to the standard Mean Squared Error (MSE), but differences in datasets, backbones, and evaluation protocols make their practical benefits difficult to compare.
By Danil Rudenko · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.