FairRSFM: A Biome-Aware Benchmark and Debiasing Framework for Remote Sensing Foundation Models
FairRSFM finds biome-level accuracy gaps in remote-sensing foundation models that aggregate scores hide.
FairRSFM is a biome-aware benchmark and debiasing framework for remote sensing foundation models. It maps 14 terrestrial biome classes into six macro-groups and evaluates frozen backbones on m-EuroSAT, m-BigEarthNet, m-SA-Crop-Type, and MMEarth20K. Across Prithvi-EO-2.0, SatMAE, and DOFA, overall metrics mask worst-group gaps, including Prithvi-EO-2.0 dropping from 90.98% macro-F1 to 83.72% worst-group on m-EuroSAT. BOLP, Dynamic Biome Reweighting, and GroupDRO help in a model- and task-dependent way; BOLP lifted Prithvi worst-group F1@opt on m-BigEarthNet from 46.12% to 50.27%.
- Fourteen biomes are mapped into six ecological macro-groups.
- Frozen backbones are tested on four remote-sensing datasets.
- Aggregate scores hide worst-group drops for Prithvi-EO-2.0, SatMAE, and DOFA.
- BOLP raised one Prithvi worst-group score without updating the backbone.
- AI models
- Prithvi-EO-2.0SatMAEDOFA
From the original article
1 min read at huggingface.co
Remote sensing foundation models (RSFMs) are commonly evaluated using aggregate metrics, which can hide systematic performance disparities across ecological regions. We introduce FairRSFM, a biome-aware benchmark for evaluating ecological group robustness in RSFMs.
By Md Aminur Hossain, Omkumar Vaghasiya, Rajeev Ranjan Dwivedi · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.