Reframe aligns visual agents on tool observations during distillation
Original title: Reframe: Observation-Aligned Multi-Teacher On-Policy Distillation for Agentic Visual Reasoning
GD-GRPO and observation-gated multi-teacher distillation beat Mix-RL and vanilla MOPD on 14 benchmarks.
Reframe is a post-training framework for agentic visual reasoning that aligns policies on observation state, not only output distributions. Ground-Drop-GRPO trains spatial and temporal specialists with evidence-aware rewards and stochastic removal of the original visual input. Observation-Gated-MOPD replaces deficient tool actions mid-rollout and resumes distillation from the repaired observation, without a task-specific SFT cold start.
Unconfirmed: Benchmark gains are reported by the paper and are not independently confirmed.
- Neither stage requires a task-specific supervised fine-tuning cold start.
- Ablations say GD-GRPO raises observation dependence versus outcome-based GRPO.
- The method covers both image and video agent benchmarks.
- Poorly grounded tool actions can limit evidence available for later distillation.
From the original article
1 min read at huggingface.co
Agentic reinforcement learning (RL) and multi-teacher on-policy distillation (MOPD) offer complementary routes to post-training. For visual agents, however, tool use presents two related challenges. Models may overlook useful tool-returned observations, while poorly grounded actions can limit the evidence available for subsequent distillation.
By Haozhi Yuan, Handong Li, Yifan Zhang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.