Conditional autoencoder erases sensitive concepts and steers them
Original title: Principled Nonlinear Concept Erasure and Steering
Under stated assumptions, perfect reconstruction plus independence implies nonlinear guardedness, without task labels.
The paper proposes a conditional autoencoder that removes a sensitive attribute, such as gender or race, while keeping other content. It argues that perfect reconstruction together with population-level independence is enough for perfect nonlinear guardedness when assumptions hold, with weaker preservation guarantees if reconstruction is imperfect. Empirically it reports a better utility-erasure trade-off than supervised competitors despite using no target-task labels, and inference-time concept conversion by changing only the target concept.
Unconfirmed: Named datasets, attributes beyond the examples, and language models are not listed in the abstract.
- Linear erasure methods are guaranteed only against linear probes.
- The method uses no target-task labels and converges faster, the authors say.
- Steering needs no extra training beyond the erasure model.
- Tests cover multiple datasets, attributes, and language models.
From the original article
1 min read at arxiv.orgLearned representations often encode sensitive attributes, such as gender or race, that may be undesirable or unsafe to retain. Concept erasure aims to remove this information while preserving task-relevant content.
By Idan Simai · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.