Decoy Direction Optimization: A Post-Hoc Defense Against LLM Abliteration
Researchers introduce Decoy Direction Optimization, a cheap weight-editing defense that blinds refusal-direction ablation attacks against open-weight LLM safety guardrails.
Refusal Feature Ablation bypasses safety guardrails in open-weight LLMs by projecting out a linear refusal direction, often with high attack success rates. Decoy Direction Optimization injects a high-magnitude nonlinear decoy into MLP neurons so attackers' contrastive estimators ablate a harmless orthogonal feature instead. Evaluated across six model families, DDO keeps ASR below 10% under standard RFA and on Llama-3-8B-Instruct reduces Heretic weight-level attack ASR from 88.7% to 18%. It costs 30 to 450 times less per configuration than trained defense baselines.
- DDO injects nonlinear decoy signals into MLP neurons, corrupting contrastive estimators used to find refusal directions.
- Achieves under 10% attack success rate against standard Refusal Feature Ablation across six model families.
- On Llama-3-8B-Instruct, cuts Heretic weight-level attack ASR from 88.7% to 18%.
- Runs 30 to 450 times cheaper per configuration than safety-finetuned baseline defenses.
From the original article
1 min read at huggingface.co
Safety guardrails in open-weight language models can be readily bypassed using Refusal Feature Ablation (RFA), a technique that identifies and projects out a linear refusal direction from the residual stream, often achieving a high attack success rate (ASR) while preserving model capability. Defending against these attacks typically requires computationally expensive safety finetuning for every new checkpoint.
By Aashiq Muhamed, Mona T. Diab, Virginia Smith · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.