AttentionSwitch: Joint Learning for LLM Jailbreak Defense
Original title: AttentionSwitch: Jailbreak Defense for LLMs via Joint Learning of Sparse Attention-Head Selection and Scaling
AttentionSwitch jointly optimizes sparse attention head selection and query-conditioned scaling to defend LLMs against jailbreaks without catastrophic forgetting.
Large language models remain susceptible to jailbreak attacks that circumvent safety mechanisms. Existing defenses typically decouple intervention site selection from modification strength, which can be suboptimal. AttentionSwitch is the first method to jointly learn a sparse set of attention heads together with multiplicative scaling factors on frozen LLMs, allowing attenuation or amplification of selected heads without altering representation directions. Across popular language models and jailbreak attacks, it outperforms existing defenses, with particularly strong results when training data is scarce.
- First joint optimization of intervention sites and strengths on frozen LLMs
- Learns sparse attention heads with query-conditioned scaling factors
- No change to representation directions in selected heads
- Outperforms prior defenses under data-scarce conditions
- Applies to popular language models and multiple attacks
- AI models
- LLMs
From the original article
1 min read at arxiv.orgLarge language models have achieved remarkable performance across a wide range of tasks, yet they remain susceptible to jailbreak attacks that circumvent safety mechanisms and elicit harmful or unintended outputs. Although safety can be improved through model fine-tuning, such approaches can be computationally expensive and may degrade previously acquired capabilities through catastrophic forgetting.
By Doniyorkhon Obidov · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.