Distillation Defenses Easily Break After Reinforcement Learning
Distillation defenses for closed-source LLMs can fail after later reinforcement learning.
The paper argues that distillation defenses are typically evaluated only immediately after distillation, ignoring later reinforcement learning. Further RL training can break defenses that appeared effective and lowers the cost of copying closed-source reasoning. The authors report that simple attacks using data obtainable from current APIs produce reasoning gains comparable to more sophisticated extraction of full hidden traces. They conclude that defenses leaking enough information to reconstruct approximate reasoning traces are likely ineffective and discuss batch-level alternatives.
- Reinforcement learning after distillation can break defenses that initially looked effective.
- API-obtainable data can match gains from extracting full hidden reasoning traces.
- Defenses that leak reconstructable reasoning traces are argued to be ineffective.
- Authors discuss batch-level controls as a potentially stronger defense.
From the original article
1 min read at arxiv.orgDistillation attacks copy the reasoning capabilities of closed-source large language models, allowing bad actors to replicate state-of-the-art performance at low cost. Attackers systematically collect a large volume of frontier model reasoning traces and then train (i. e.
By Shidan Javaheri · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.