DecisionUnlearn-Bench measures unlearning when decision schemas change
Original title: DecisionUnlearn-Bench: Unlearning Probabilistic Decisions under Changing Schemas
On BANKING77, full-to-oracle displacement rose from example deletion to class deletion across 20 configurations.
DecisionUnlearn-Bench compares full, unlearned and retain-only probabilistic decision models across deletion granularities, label renderings and candidate sets. On BANKING77, displacement from the full model to a retrained oracle increased from example to association to class deletion. Replacing semantic option labels with opaque codes nearly removed the oracle's recognition of deleted classes. SimNPO met the screening criterion in 19 of 20 configurations, but gains varied, and CLINC150 tuning showed no positive pooled effect across 48 intents.
Unconfirmed: SimNPO passed screening in most setups, but the paper says gains vary substantially by seed and target.
- Changing the candidate set moved both the oracle and measured progress.
- At matched suppression, methods differed markedly in oracle fidelity.
- Recovery tests separated initial forgetting from later adaptation.
- Conclusions depended on both model architecture and the metric used.
From the original article
1 min read at arxiv.orgMachine unlearning aims to reproduce the behavior of a model trained without designated data. For probabilistic decision models, this target also depends on the candidate options supplied at inference. We introduce DecisionUnlearn-Bench, an empirical framework that compares full, unlearned, and retain-only models across deletion granularities, label renderings, and candidate sets.
By Tiantong Wu · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.