Current probabilistic time series foundation models show systematic miscalibration in uncertainty estimates
Original title: Evaluating Uncertainty Calibration in Probabilistic Time Series Foundation Models
Probabilistic TSFMs exhibit distinct miscalibration patterns that standard metrics obscure, with post-hoc recalibration improving some but not all forms of calibration.
Time series foundation models are used for probabilistic forecasting across diverse domains, yet the reliability of their uncertainty estimates remains poorly understood. We introduce a taxonomy of how TSFMs represent predictive uncertainty and develop a fine-grained evaluation framework for calibration in multi-horizon forecasting that distinguishes pooled, horizon-wise, and stronger conditional notions of calibration. Using this framework, we conduct a comprehensive empirical evaluation of popular TSFMs across diverse datasets and show that current models exhibit distinct and systematic patterns of miscalibration that standard metrics often obscure. We further analyze common post-hoc recalibration methods and characterize which forms of calibration they can improve and which tend to persist.
- TSFMs show systematic miscalibration patterns obscured by standard metrics.
- Evaluation distinguishes pooled, horizon-wise, and conditional calibration.
- Post-hoc recalibration methods improve certain forms but not all.
From the original article
1 min read at arxiv.orgTime series foundation models (TSFMs) are increasingly used for probabilistic forecasting across diverse domains, yet the reliability of their uncertainty estimates remains poorly understood. In this work, we study uncertainty calibration in probabilistic TSFMs from a broader methodological and empirical perspective.
By Thomas Decker · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.