Addition by Subtraction strengthens LLM safety alignment against pruning and jailbreaks
Original title: Addition by Subtraction: Toward Mitigating Fragile Safety Alignment in Large Language Models
Addition by Subtraction removes safety-critical neurons during finetuning to produce models robust to sparsity-based attacks and black-box jailbreaks.
LLM safety alignment can be undermined by pruning small numbers of neurons or ablating refusal directions. Addition by Subtraction gradually removes identified safety-critical neurons during finetuning and uses preference optimization to re-establish aligned behavior. The resulting models are more robust to pruning-based attacks and generalize to other sparsity interventions and black-box jailbreaks. Experiments across four models from three vendors reduce attack success rates by up to 90 percent while improving truthfulness and largely preserving capabilities.
- Removes safety-critical neurons during finetuning
- Uses preference optimization to restore alignment
- Substantially more robust to pruning attacks
- Generalizes to refusal-direction ablation
- Reduces attack success rates up to 90 percent
From the original article
1 min read at arxiv.orgLarge language models (LLMs) undergo safety alignment to prevent harmful generations, yet they remain vulnerable to jailbreak attacks. Recent work reveals a deeper structural weakness: safety alignment can be highly sparse, such that pruning a small number of safety-critical neurons or ablating a single refusal direction can substantially undermine aligned behavior.
By Hamid Dashtbani · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.