SpatialSpeak: QA-Native Reconstruction with Local and Global Context for Spatial Chain-of-Thought Reasoning
SpatialSpeak pretrains vision-language models on multi-view reconstruction, reaching a 62.8 ReVSI spatial-reasoning score.
SpatialSpeak is a two-stage vision-language framework for multi-view spatial reasoning. Stage I, QA-Native Reconstruction Pretraining, learns local geometry via marked-point 3D queries and global scene context via object-center queries, both as text question answering. Stage II trains spatial chain-of-thought with visual compensation so the model states geometric estimates, checks reliability, and refines answers. On ReVSI, pretraining raises the CoT gain from 2.6 to 6.9 points; SpatialSpeak scores 62.8, 8.7 above the strongest baseline, and leads VSI-Bench and SPAR-Bench.
- QA-native reconstruction pretraining shares an autoregressive interface with later reasoning.
- Local marked-point and global object-center queries both improve results.
- CoT-VC gain on ReVSI rises from 2.6 to 6.9 points after pretraining.
- ReVSI score of 62.8 exceeds the strongest baseline by 8.7 points.
From the original article
1 min read at huggingface.co
Vision-language models (VLMs) can benefit from geometric priors for multi-view spatial reasoning, yet answer-only training does not directly supervise the intermediate geometric estimates and their use in deriving quantitative spatial answers.
By Yang Cao, Jiaxin Zhang, Dave Zhenyu Chen · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.