Data Unlearning via Inverse Distillation
Inverse Distillation Unlearning distills diffusion and flow models into one-step generators while suppressing a designated forget set.
Inverse Distillation Unlearning jointly distills a multi-step flow or diffusion teacher into a one-step student and suppresses outputs tied to a designated forget subset. It requires only the pretrained full-data teacher and forget-set samples, without retained training examples, extra feature extractors, or classifiers. Experiments on MNIST and CIFAR-10 under flow-matching and score-based diffusion show fewer generations of forgotten classes while quality on retained classes is preserved. The authors present it as the first unified unlearning-and-distillation framework for unconditional flow-matching and score-based models.
- IDU distills multi-step teachers into one-step generators while unlearning.
- It needs only the teacher and forget-set data, not retained examples.
- MNIST and CIFAR-10 tests cover flow-matching and score-based diffusion.
- Forgotten-class frequency drops while retained-class quality is preserved.
From the original article
1 min read at huggingface.co
Multi-step matching models, including flow and diffusion models, produce high-quality outputs but incur substantial inference costs and may reproduce unwanted components of their training datasets.
By Aleksei Leonov, Nikita Kornilov, Zhenhe Zhang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.