AEGIS: Audio Endogenous Guarding via Internal Signals Against Large Audio-Language Model Jailbreaks
AEGIS uses mid-layer risk signals to cut audio-model jailbreak unsafe rates from 17.9% to 0.4%.
The AEGIS paper argues that audio jailbreaks against large audio-language models succeed because an internal risk signal fails to become a refusal, not because the model never recognizes harmful intent. Layer-wise probing finds risk-related information remains decodable in intermediate representations, a discrepancy the authors call the risk-to-refusal gap. Their detect-then-intervene defense uses a mid-layer risk gate to selectively activate downstream safety adapters. Across six LALMs and three audio jailbreak benchmarks, average unsafe responses fell from 17.9% to 0.4%, with only a marginal increase in over-refusal on benign inputs.
- Probing shows harmful intent stays encoded, but later layers still fail to refuse.
- A mid-layer risk gate activates safety adapters only when risk is detected.
- Average unsafe rate fell from 17.9% to 0.4% across six models and three benchmarks.
- Over-refusal on benign audio increased only marginally, the authors report.
Full article141 words · extracted from arxiv.org · click to collapse
Large audio-language models (LALMs) expand language models to process and interpret audio, but also expose them to heterogeneous audio jailbreaks. We ask whether successful jailbreaks reflect failures to recognize harmful intent or failures occurring after such recognition. Layer-wise probing reveals the latter: risk-related information remains decodable from intermediate representations, yet the internal risk signal fails to translate into refusal in later-layer processing. We identify this discrepancy as the risk-to-refusal gap. Building on this finding, we propose AEGIS, a detect-then-intervene defense whose mid-layer risk gate selectively activates downstream safety adapters. Across six LALMs and three heterogeneous audio jailbreak benchmarks, AEGIS reduces the average unsafe rate from 17.9% to 0.4%, while causing only a marginal increase in over-refusal on benign inputs. These results establish selective internal intervention as an effective path toward more robust refusal in LALMs. The code is available at https://github.com/azzzzliao/aegis-audio-defense.
Text extracted automatically; images, tables and formatting may be missing. Original: https://arxiv.org/abs/2609.29287