Probes test spatial state in ten visual foundation models
Original title: Probing Spatial State Representations in Visual Foundation Models
Strong visible-scene readout does not ensure object persistence, future-feature retrieval or reliable action utility.
The paper asks whether visual foundation models support a persistent spatial state under self-motion. Frozen-feature probes evaluate ten representations on four levels—spatial structure, object persistence, predictive spatial update and actionability—using streaming egocentric views of static indoor environments. Capability profiles vary by task, layer and prefix length. Object persistence and actionability build on camera-pose readout, while feature prediction builds on visible-scene readout.
Unconfirmed: The abstract does not name the ten representations or report numeric probe scores.
- Matched controls separate readout accuracy from reliance on visual history and motion.
- A spatial reader trained on real features does not transfer unchanged to predicted features.
- Predicted actions respond to goal changes, but the correct goal does not reliably improve progress.
- Native future prediction still retains recoverable spatial structure.
From the original article
1 min read at arxiv.orgAn observer sees only part of the surrounding world, yet must keep track of what lies beyond the current view. Spatial perception therefore involves more than recognizing individual images: it calls for a persistent spatial state that is updated under self-motion and used to guide action. Do visual foundation models provide representations that support such a state?
By Baiqiao Yin · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.