Visual-Attribution Gating filters contaminated corrections in visual RL
Original title: Reinforcement Learning with Visually Attributable On-Policy Self-Distillation
On six visual reasoning benchmarks, VAG beats GRPO and on-policy self-distillation on average.
The paper studies multimodal reinforcement learning with verifiable rewards and on-policy self-distillation for visual reasoning. It defines privileged-correction contamination, where token-level teacher corrections cannot be consistently attributed to privileged visual evidence. Visual-Attribution Gating uses the verifier-derived advantage as the optimization anchor and keeps only corrections supported by the contrast between evidence-present and evidence-degraded teacher views. Across six visual reasoning benchmarks, the authors report that VAG consistently outperforms GRPO and OPSD and sets a new average state of the art.
Unconfirmed: The state-of-the-art claim is the authors’ and is not independently confirmed here.
- VAG admits only corrections supported by an evidence-present versus evidence-degraded contrast.
- The optimization anchor is the verifier-derived advantage.
- Results are reported on six visual reasoning benchmarks.
- Compared methods are GRPO and on-policy self-distillation.
From the original article
1 min read at arxiv.orgMultimodal reinforcement learning with verifiable rewards (RLVR) provides reliable task-level supervision, yet offers limited guidance for localizing errors in visual reasoning. On-policy self-distillation (OPSD) complements this supervision with dense token-level corrections from a privileged teacher equipped with an evidence-centered visual view.
By Shiao Xie · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.