Rethinking Latent Visual Reasoning: Grounding Latent Reasoning in Visual Evidence
ReaLVR adds visual-evidence supervision to latent reasoning, hitting 63.7% on Qwen2.5-VL-7B and scaling to 235B.
The paper analyzes latent visual reasoning, where multimodal models compute in continuous latent tokens, and finds a latent evidence-credit gap: tokens respond only weakly to image changes that alter the correct answer. The authors argue GRPO with only a final-answer reward does not say which visual evidence to preserve. Their method, ReaLVR, supervises free-running latent trajectories by contrasting correct and model-generated wrong answers and relevant versus mismatched visual evidence. Across three model families it beats evaluated LVR baselines, with a five-task average of 63.7% on Qwen2.5-VL-7B, and gains continue at scales up to 235B.
- Latent tokens respond weakly to answer-changing image perturbations.
- Final-answer GRPO rewards give little visual-evidence supervision.
- ReaLVR reaches a 63.7% five-task average on Qwen2.5-VL-7B.
- Improvements hold across three families and scales up to 235B.
- Organizations
- Alibaba
- AI models
- Qwen2.5-VL-7B
From the original article
1 min read at huggingface.co
Latent visual reasoning (LVR) enables multimodal large language models (MLLMs) to perform intermediate computation in continuous latent tokens rather than expressing every reasoning step in words. However, unlike textual CoT, latent reasoning is not directly observable, making it difficult to supervise what latent tokens learn.
By Xi Xiao, Tianchen Zhao, Youngeun Kim · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.