Agentic-OPD recovers forgotten agent skills from discarded RL checkpoints
Authors report gains of up to 7.4 points across 24 benchmarks, with a 4B student beating GPT-5.6 on deep search.
Stories per day · 30 days
4 RSS
Relatedreinforcement-learning5qwen33decision-models2fp82grpo2kv-cache2llm-agents2quantization2retrieval2teacher-student2tool-use2actionopd1agent-distillation1agentic-ai1agentic-distillation1agentic-rl1agents1compute-efficiency1
40 stories, newest first · one card per story
Stories tagged #on-policy-distillation
Reframe aligns visual agents on tool observations during distillation
GD-GRPO and observation-gated multi-teacher distillation beat Mix-RL and vanilla MOPD on 14 benchmarks.
Paper measures mismatch on-policy distillation can tolerate
Model-specific FP8 recipes kept quality and fit at least 1.93x as many KV-cache tokens as BF16.
Strong-to-weak OPD achieves 50x compute efficiency versus GRPO when using own checkpoint
On-policy distillation with the student's own GRPO checkpoint reaches target accuracy with 50 times fewer FLOPs than GRPO training.
DecisionRank on-policy distillation beats generative teacher on retrieval tasks
Item-space OPD lets 1.7B DecisionRank surpass 27B LLM teacher on TREC DL while running 24.5x faster.
DecisionRank trains decision model rerankers
DecisionRank achieves SOTA on TREC DL and BEIR with a 1.7B model that surpasses its 27B generative teacher.
Reward-free repair targets myopic scoring in agentic distillation
R2S beat three agentic on-policy methods in all 12 teacher-student-environment cells, by 12.8 success points on average.
SimOPD: Relational Knowledge Transfer for On-Policy Distillation
SimOPD distills teacher knowledge through response similarities, gaining up to 3.1 points cross-architecture.
Waypoint-OPD: Transferring RL-Induced Policy Shifts via Gated On-Policy Distillation
Waypoint-OPD gates RL policy-shift distillation and leads math and code scores for Qwen3 students.
MMSearch-OPD: Easy Made Hard, Hard Made Learnable for Long-Horizon Multimodal Search Agents
MMSearch-OPD hardens multimodal questions and jointly trains long-horizon search agents with RL and distillation.
Recovering Reliable Supervision via Critique-Guided On-Policy Distillation
CriticOPD uses teacher critiques to restore recoverability in on-policy distillation, lifting two Qwen3 students on math benchmarks.
ActionOPD: Distilling Decisions, Not Tokens
ActionOPD distills black-box teacher decisions into Qwen3 agents, gaining 3.3-6.9 points out of distribution.
Branching On-Policy Distillation
BranchOPD branches uncertain on-policy rollouts, beating vanilla OPD by 18.55 math and 13.8 ALFWorld points.
Rethinking Relative Policy Differences Transfer of On-Policy Distillation
Relative policy differences, not expert strength alone, determine useful on-policy distillation transfer.
When Do We Need On-Policy Distillation? Distilling on Offline Student Rollouts Is Often Better
Semi-OPD, distilling on offline student rollouts, beats on-policy distillation in 14 of 17 teacher-student pairs.
A Good Self-Teacher Meets the Student Where They Are: Joint On-Policy Learning and Teaching
JOLT jointly trains a privileged self-teacher and student so on-policy distillation matches the student's ability.
On-Policy Distillation Teaches New Skills but Not New Knowledge
On-policy distillation transfers compositional reasoning skill but little new factual knowledge under reverse KL.
On-Policy Distillation with Negative-Policy Rollouts
NP-OPD improves on-policy distillation by using weaker-model rollouts as an explicit negative signal.
Sharpen Without Search: On-Policy Distillation of Sequence-Level Power Distribution
On-policy power distillation trains models to sample sharpened answers once, lifting MATH500 by up to 23 points.
Gains and Collapse in On-Policy Distillation:A Reinforcement Learning Perspective
On-policy distillation can boost language models or collapse them into repetitive reward hacking.
Harness-Aware Distillation for Small Language Model Agents
Harness-Aware Distillation trains smaller agents on skills the fixed software harness cannot supply.
OPD Before RL: Warm-Starting Rubric-Based RL with On-Policy Distillation
Rubric-privileged on-policy distillation before RL beat SFT-then-RL on health and science benchmarks.
PivotOPD: Learning to Recover from Pivotal Mistakes in Multi-Turn Agents
PivotOPD trains agents to avoid and recover from early pivotal mistakes, lifting ALFWorld and SWE-Bench scores.
PivotOPD: Learning to Recover from Pivotal Mistakes in Multi-Turn Agents
PivotOPD trains multi-turn agents to avoid pivotal mistakes and recover within a few turns.