AEGIS: Audio Endogenous Guarding via Internal Signals Against Large Audio-Language Model Jailbreaks
AEGIS uses mid-layer risk signals to cut audio-model jailbreak unsafe rates from 17.9% to 0.4%.
The AEGIS paper argues that audio jailbreaks against large audio-language models succeed because an internal risk signal fails to become a refusal, not because the model never recognizes harmful intent. Layer-wise probing finds risk-related information remains decodable in intermediate representations, a discrepancy the authors call the risk-to-refusal gap. Their detect-then-intervene defense uses a mid-layer risk gate to selectively activate downstream safety adapters. Across six LALMs and three audio jailbreak benchmarks, average unsafe responses fell from 17.9% to 0.4%, with only a marginal increase in over-refusal on benign inputs.
- Probing shows harmful intent stays encoded, but later layers still fail to refuse.
- A mid-layer risk gate activates safety adapters only when risk is detected.
- Average unsafe rate fell from 17.9% to 0.4% across six models and three benchmarks.
- Over-refusal on benign audio increased only marginally, the authors report.
From the original article
1 min read at arxiv.orgLarge audio-language models (LALMs) expand language models to process and interpret audio, but also expose them to heterogeneous audio jailbreaks. We ask whether successful jailbreaks reflect failures to recognize harmful intent or failures occurring after such recognition.
By Yu-Ling Liao · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.