Where-OPD: Spatially Guided On-Policy Self-Distillation of MLLMs with Synthetic Scenes
Where-OPD uses spatial self-distillation on synthetic scenes to improve multimodal perception.
Where-OPD is an on-policy self-distillation method for multimodal large language models that gives the teacher textual, spatially grounded guidance about relevant visual elements. Procedurally generated scenes provide object identities and coordinates without human annotations or external teachers. Although post-training uses only synthetic scenes, the student improves counting, document, and chart tasks and gains 3.23 points on average across CVBench, V*, ZoomBench, BLINK, HR-Bench, and MME-RealWorld.
- Teacher gets textual spatial guidance; student sees image and question
- Procedural scenes supply objects and coordinates without human labels
- Gains cover counting, document, and chart understanding
- Synthetic-only training adds 3.23 points on six real-world benchmarks
From the original article
1 min read at huggingface.co
On-policy self-distillation has recently emerged as an effective approach for improving language-model reasoning by supervising students with a frozen or EMA version of themselves that receives privileged information. Its application to multimodal large language models (MLLMs), however, remains largely unexplored.
By Sophia Sirko-Galouchenko, Monika Wysoczanska, Andrei Bursuc · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.