Theory separates locality from weight sharing in convolutional learning
With uncorrelated inputs a convolutional student learns exactly P times faster, but spatial correlation can erase that edge.
Stories per day · 30 days
2 RSS
Relatedon-policy-distillation3qwen32agentic-distillation1convolutional-networks1criticopd1distillation1learning-dynamics1llama1llm-agents1llm-training1mathematical-reasoning1multi-task1on-policy1post-training1reinforcement-learning1reward-modeling1sample-complexity1scout1
5 stories, newest first · one card per story
Stories tagged #teacher-student
Reward-free repair targets myopic scoring in agentic distillation
R2S beat three agentic on-policy methods in all 12 teacher-student-environment cells, by 12.8 success points on average.
Recovering Reliable Supervision via Critique-Guided On-Policy Distillation
CriticOPD uses teacher critiques to restore recoverability in on-policy distillation, lifting two Qwen3 students on math benchmarks.
On the Off-Policy Teacher in On-Policy Distillation
SCOUT adapts teachers to student-generated prefixes and improves on-policy distillation across scales and domains.
DuoOPD: Learning from Joint Teacher-Student Outcomes for Multi-Task On-Policy Distillation
DuoOPD uses joint teacher-student outcomes to improve multi-task on-policy distillation over standard OPD.