OP-Print targets membership inference in on-policy self-distillation
Original title: OP-Print: Tracing Membership Imprints in On-Policy Self-Distillation
Gray-box and black-box signals use sparse correction imprints, while the authors say baseline attacks stay near chance.
A cs.CR paper presents OP-Print, a membership-inference framework for on-policy self-distillation, where a student is supervised by its own outputs plus extra context. The authors argue the supervision imprint is sparse and concentrated on uncertainty-sensitive correction sites. In gray-box access, the attack aggregates teacher-student residual corrections; in black-box access, it uses restart-based output-basin stability. They report that baseline attacks remain near random guessing even with gray-box access, while OP-Print works in both settings across models and schemes they tested.
Unconfirmed: Effectiveness is the authors' evaluation, not an independently confirmed attack.
- The study covers gray-box and black-box threat models.
- Gray-box OP-Print aggregates privileged residual corrections along one rollout.
- Black-box OP-Print measures restart-based output-basin stability.
- Authors say baseline attacks stay near random guessing under gray-box access.
- Effectiveness is reported across diverse models and distillation schemes.
Impact: Infer whether an example was used in self-distillation
From the original article
1 min read at arxiv.orgOn-policy distillation has been widely adopted for alleviating the distribution mismatch of off-policy distillation. Self-distillation further removes the need for an external teacher by using the student's own outputs, enhanced with extra context or demonstrations, as supervision. This teacher-free design makes on-policy self-distillation practical, yet its privacy risks remain largely unexplored.
By Qi Li · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.