EarthRecord tests agents on four decades of raw satellite imagery
Original title: EarthRecord: From Pixels to Land Change with Multimodal Agents over Four Decades of Satellite Imagery
Twelve agents mostly finish analyses, but half of correct answers follow an altered satellite when the ground is unchanged.
EarthRecord is a benchmark of 56 sites and 238 land-change questions over Landsat, MODIS, VIIRS and Sentinel-2 archives, leaving cloud masking and cross-sensor harmonization to the agent. Scores use withheld global products, agreement among methods and visual interpretation. Closed-book and instrument-pair controls show answers move from prior knowledge toward the pixels, but not reliably to real ground change.
Unconfirmed: Agent scores are the paper's evaluation of twelve multimodal agents.
- The record spans four decades and ten analysis tasks.
- Leading agents falter when an answer requires mapping from imagery.
- Altering one satellite's data caused half of previously right answers to follow the change.
- A closed-book control removes the satellite record.
From the original article
1 min read at huggingface.co
While multimodal agents now carry out complex data analysis with tools, their ability to turn decades of satellite imagery into reliable land-change analysis, crucial for Earth observation science, remains unmeasured. Existing benchmarks fall short: they pose questions on curated images or give agents preprocessed inputs, leaving out the raw multi-sensor archives where such analysis begins.
By Haoran Zhang, Huizhen Sun, Hanbin Wen · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.