P^2-MOPD accelerates multi-teacher distillation with parameter priors
P^2-MOPD jointly optimizes mixing coefficients and student weights to reach 94.7% recovery in half the steps of MOPD.
Stories per day · 30 days
2 RSS
Relateddistillation4self-distillation3llm-training2reinforcement-learning2language-models1llm-agents1mathematical-reasoning1membership-inference1model-privacy1multi-agent1multi-teacher1parameter-merging1policy-distillation1question-generation1qwen31ranking1synthetic-data1
31 stories, newest first · one card per story
Stories tagged #on-policy
Order-Only On-Policy Distillation improves LLM students
O^{3}PD trains students to match teacher token ordering without probability values or temperature tuning.
MTA-OPSD lets each token pick its own teacher in self-distillation
On Qwen3 math tasks, the method beats prior on-policy self-distillation, with gains of up to 5.9 over the base model.
OP-Print targets membership inference in on-policy self-distillation
Gray-box and black-box signals use sparse correction imprints, while the authors say baseline attacks stay near chance.
AOPD weights teacher actions by student advantage in on-policy distillation
The estimator needs only student rollouts and terminal rewards, with no extra rollouts or reward evaluations.
Ask-to-Distill learns to write questions for on-policy distillation
A frozen student–teacher pair scores synthetic questions without reference answers, then those prompts feed standard on-policy distillation.
MultiAgent-OPD jointly trains a hierarchical agent team end to end
Across seven datasets the paper reports 38.68% macro avg@4, above Prompted Critic and C-MAS + RL.
NVIDIA PivotOPD Teaches Multi-Turn AI Agents to Recover From Pivotal Mistakes
NVIDIA's PivotOPD trains multi-turn LLM agents to avoid and recover from early pivotal mistakes.
Composing What Each Teacher Learned: Multi-Teacher On-Policy Distillation through Teacher-Relative Shifts
Δ-MOPD transfers teacher-minus-base logit shifts, beating endpoint multi-teacher distillation on math and mixed benchmarks.
Rethinking Cross-Tokenizer On-Policy Distillation: From Alignment Coverage to Supervision Reliability
Cross-tokenizer on-policy distillation works better with strict token alignment than broader, less reliable supervision.
Internalizing Agent Experience into Diffusion Model Weights via On-Policy Context Distillation
D-OPCD distills a text-to-image agent harness into diffusion weights, lifting scores from 60.52 to 65.09.
DiffGate: Difficulty-Gated Teacher Guidance for On-Policy Distillation
DiffGate gates bounded teacher guidance to failed rollouts, lifting Qwen3 pass rates over matched GRPO.
From Gradients to Capabilities: Understanding Multi-Teacher On-Policy Distillation
A study of Qwen3-1.7B shows multi-teacher on-policy distillation is shaped by averaging, Adam, and BF16 precision.
From Gradients to Capabilities: Understanding Multi-Teacher On-Policy Distillation
A distillation study shows averaging, Adam, and BF16 reshape how multi-teacher signals update student models.
Latent-MOPD: Latent Multi-Teacher On-Policy Distillation
Latent-MOPD distills multiple LLM specialists via hidden states and token predictions, beating baselines on nine benchmarks.
OPMD distills teacher intermediate computation into on-policy students
The paper reports the highest average accuracy among distillation baselines in all three teacher-student settings.
Overcoming Scaling Limits in On-Policy Self-Distillation for LLM Reasoning
OASIS keeps on-policy self-distillation effective at scale by supervising verified reasoning trajectories.
MemFold: Learning Compact Soft Memory for Long-Context Personalization via On-Policy Optimization
MemFold trains fixed-size soft memory with on-policy rewards and leads PersonaMem personalization across Qwen backbones.
Beyond Teacher Assignment: Domain-Normalized Multi-Teacher On-Policy Distillation
Domain-normalized multi-teacher distillation rebalances specialist feedback so Qwen3.5 students retain mathematics gains.
Learning from Teacher Continuations at Student States
OLIVE updates a student on teacher continuations of student-generated prefixes, beating offline distillation under the same budget.
PMOPD: Task Ordering, Cycling, and Parameter-Update Subspace Protection in Multi-Teacher On-Policy Distillation
PMOPD reduces cross-task interference in multi-teacher distillation, improving Code, Reason, and Math on Qwen and Llama.
On-Policy or Off-Policy Learning? A Systematic Study of Distillation Dynamics
Controlled distillation finds KL direction matters more than on-policy versus off-policy rollouts.
On-Policy Self-Distillation for Multi-Turn Image Editing
MT-OPSD uses on-policy self-distillation so image editors stay stable across iterative multi-turn edits.
Do We Really Need KL Divergence for On-Policy Distillation of Large Language Models?
Researchers show on-policy LLM distillation works without KL divergence if updates move toward the teacher.