SpatialASM: Internalizing Active Mentalization for Spatial Reasoning
SpatialASM trains vision-language models to plan viewpoints and simulate scenes for spatial reasoning.
SpatialASM is a training framework for giving large vision-language models Active Spatial Mentalization, the ability to revisit a scene from task-relevant viewpoints and reconstruct its layout. An allocentric novel-view generator grounds hypothetical viewpoints in visual evidence and supplies explicit reasoning trajectories plus automatically built perspective-taking tasks. On seven benchmarks spanning perception-oriented understanding through multi-image and viewpoint-transformation reasoning, the method improved average accuracy across four LVLM backbones while retaining general vision ability. The authors also present synthesized trajectories as a way to expand scarce spatial chain-of-thought supervision.
- SpatialASM teaches vision-language models active viewpoint planning.
- Training uses simulated views and explicit spatial reasoning trajectories.
- Average accuracy rose on seven benchmarks and four LVLM backbones.
- General vision performance was retained after spatial training.
From the original article
1 min read at huggingface.co
When solving complex spatial reasoning tasks, ordinary humans naturally revisit the scenes from task-relevant viewpoints and mentally reconstruct the spatial layouts from those viewpoints. But can Large Vision-Language Models (LVLMs) proactively think in the same way?
By Yuhong Liu, Shenxi Wu, Junyi Chi · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.