CycleGRPO couples region captioning and localization with cycle rewards
Original title: Generalized CycleGRPO for Fine-Grained Multimodal Understanding and Grounding
On GroundingSuite, SAMTok overall gIoU rises from 57.5% to 68.2%; video tIoU on Charades reaches 51.9% versus 45.6% for Qwen3-VL.
The paper introduces Generalized CycleGRPO, a reinforcement-learning setup that trains multimodal models to describe a designated region and then localize that description back to the same region. Reconstruction overlap is the captioning reward, and the same region-space cycle covers image masks and boxes plus video intervals and timestamped spatial evidence, without reference captions or an external language-model judge. The authors also release DOVE, a human-verified contrastive benchmark for temporally distinctive video captions. Reported gains include SAMTok overall gIoU from 57.5% to 68.2% on GroundingSuite and average gIoU from 71.2% to 78.7% on GRES.
- Temporal results are 51.9% tIoU on Charades versus 45.6% for Qwen3-VL.
- DOVE Temporal Evidence Coverage is 33.3% versus 14.8% for the Qwen3-VL initialization.
- Training uses no reference captions and no external language-model judge.
- Models, code, and the DOVE benchmark are publicly linked.
From the original article
1 min read at huggingface.co
Fine-grained multimodal intelligence requires models to both understand a designated region and localize it from language. Existing methods learn these directions with separate objectives and task-specific annotations. We present Generalized CycleGRPO, the first unified reinforcement learning framework coupling region understanding and localization across spatial, temporal, and spatio-temporal domains.
By Xin Zhang, Haochen Wang, Beibei Lin · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.