Random splits overstate machine-learning rainfall skill in Bangladesh
Original title: RainCheck: Harnessing Historical Weather Data with Machine Learning for Rainfall Forecasting in Bangladesh
On 618,650 station-days, climatology already classified high-rainfall months at 88.4% accuracy, and learners added at most 0.8 points.
The paper compares random, temporal, and spatio-temporal evaluation of rainfall models using Bangladesh Meteorological Department records from 1948 to 2014. A calendar-only random forest looked strong under random splits but fell below climatology on future years. Lagged rainfall added modest next-day skill that disappeared beyond a two-day lead, and no learner beat climatology significantly for monthly totals.
- The dataset covers 35 stations and 618,650 station-days.
- Next-day Brier skill over climatology was 0.074, with a 95% CI of 0.064–0.088.
- Skill rose to 0.131 when other stations' same-day observations were included.
- A GRU matched but did not exceed gradient boosting.
- The authors release a leakage-aware evaluation protocol and code.
- Organizations
- Bangladesh Meteorological Department
- Countries
- Bangladesh
From the original article
1 min read at arxiv.orgMachine-learning rainfall forecasts are often evaluated by shuffling station records into random training and test sets. For rainfall, this lets a model see what happened at neighbouring stations on the very day it is asked to forecast, and it hides the fact that a seasonal lookup table already explains most of the variance.
By Shahran Rahman Alve · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.