Agentic-OPD recovers forgotten agent skills from discarded RL checkpoints
Original title: Agentic-OPD: Recovering Forgotten Agent Capabilities via On-Policy Distillation for Agentic Tasks
Authors report gains of up to 7.4 points across 24 benchmarks, with a 4B student beating GPT-5.6 on deep search.
The paper argues that keeping only the best-validation checkpoint in GRPO training of LLM agents discards earlier solutions. Agentic-OPD treats that checkpoint as the student and saved checkpoints as frozen teachers. Highest Coverage Next selects teachers by coverage of unsolved problems, and Locality-Aware Distillation limits transfer to failed rollouts where the teacher is better. The authors say a 4B student then surpassed GPT-5.6 and DeepSeek-V4.1 on deep search.
Unconfirmed: Benchmark gains and the claim of surpassing GPT-5.6 and DeepSeek-V4.1 are the authors' reported results, not independently verified.
- No single checkpoint dominates; union coverage exceeds any one checkpoint.
- Teachers are chosen by weighted coverage of the student's unsolved problems.
- Distillation is limited to failed rollouts where the teacher is locally better.
- Evaluation covers five agentic domains and 24 benchmarks and task subsets.
- The method is framed as post-RL recovery, orthogonal to the RL algorithm.
- AI models
- GPT-5.6DeepSeek-V4.1
From the original article
1 min read at arxiv.orgReinforcement learning is now the mainstream paradigm for training LLMs on agentic tasks, yet standard practice keeps only the best-validation checkpoint and discards the rest.
By Tingcheng Bian · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.