LoopVL: Recurrent Visual Intelligence
LoopVL extends loop transformers to vision-language models and beats similarly sized non-recurrent baselines.
LoopVL combines Module-Loop and Model-Loop computation to iteratively update a shared vision-language state. It is trained from scratch through language pre-training, multimodal training, and post-training. The model outperforms a range of similarly sized and larger non-recurrent models on multimodal understanding and visual reasoning benchmarks. The authors report Visual Aha Moments, marked by sharp shifts in visual attention across loops.
- LoopVL uses shared modules for recurrent vision-language states.
- Training covers pre-training, multimodal training, and post-training.
- It beats similarly sized and larger non-recurrent models.
- Visual attention shifts sharply across recurrent loops.
From the original article
1 min read at huggingface.co
We introduce LoopVL to study whether Loop Transformers can be effectively extended to vision- language models. LoopVL combines Module-Loop and Model-Loop computation to iteratively update a unified vision-language state through shared modules. We train LoopVL from scratch through language pre-training, multimodal training, and post-training.
By Zhe Qian, Ziyang Gong, Zhongxing Xu · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.