Why Forget-Only Unlearning Needs Memorization
A theory paper shows forget-only machine unlearning can require memorizing far more training data than ordinary learning keeps.
The paper studies forget-only unlearning, where a deletion algorithm sees only the trained model and examples to forget, not retained data. It proves that identical trained models from different datasets can require very different post-deletion outputs, yielding lower bounds on how closely unlearning can match retraining. For simple threshold learners, the information needed to handle arbitrary deletions can equal the entire dataset, even though training stores only one boundary point. The authors conclude models built for forget-only unlearning may need to retain more information than standard training discards.
- Deletion sees only the trained model and the forget examples.
- Different datasets can yield the same model but different retraining targets.
- Lower bounds limit how closely forget-only unlearning can match retraining.
- Threshold learners may need dataset-scale memory for arbitrary deletions.
From the original article
1 min read at arxiv.orgMachine unlearning asks for a deletion algorithm whose output is close to retraining from scratch without the selected forget examples. In this work, we study forget-only unlearning, where the deletion algorithm receives only the trained model and the examples to forget, with no retained data or extra training information. We ask whether forget-only unlearning is always possible.
By Luka Radić · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.