SpatialASM: Internalizing Active Mentalization for Spatial Reasoning
SpatialASM trains vision-language models to plan viewpoints and imagine scenes, improving accuracy on seven spatial benchmarks.
SpatialASM is a training framework that internalizes Active Spatial Mentalization in large vision-language models. An allocentric novel-view generator produces explicit trajectories teaching when to plan and imagine viewpoints, plus automatically constructed perspective-taking tasks. Tested on four LVLM backbones and seven benchmarks spanning perception, multi-image, cross-view, and viewpoint-transformation reasoning, it improves average spatial accuracy while retaining general vision ability. The authors present synthesized trajectories as scalable spatial chain-of-thought supervision.
- Active spatial mentalization means revisiting scenes from relevant viewpoints.
- Supervision comes from novel-view generation and perspective-taking tasks.
- Four LVLM backbones improve across seven spatial reasoning benchmarks.
- General vision ability is retained as spatial accuracy rises.
- AI models
- SpatialASM
From the original article
1 min read at arxiv.orgWhen solving complex spatial reasoning tasks, ordinary humans naturally revisit the scenes from task-relevant viewpoints and mentally reconstruct the spatial layouts from those viewpoints. But can Large Vision-Language Models (LVLMs) proactively think in the same way?
By Yuhong Liu · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.