CASCADE Against Jailbreaks: Combination Across Stages with Controlled Attack-Defense Evaluation
A controlled study finds layered jailbreak defenses beat any single defense across 19 attacks.
CASCADE is a systematic evaluation of how to combine LLM jailbreak defenses within and across pipeline stages, including input modification and output guards. The authors use one threat model of direct, black-box, single-turn attacks, with a standardized attack-success rate, controlled query budgets, and explicit fairness rules. Across 19 attacks and 15 defenses, no single defense is universally best, but selected combinations improve safety with little utility loss.
- Studies defense combinations within and across LLM pipeline stages.
- Uses one black-box single-turn threat model and fixed query budgets.
- Tests 19 attacks against 15 defenses under shared success criteria.
- No universal defense; chosen stacks cut risk with little utility loss.
Full article134 words · extracted from arxiv.org · click to collapse
Defenses against jailbreak attacks on Large Language Models (LLMs) operate at different pipeline stages, such as input modification or output guard, but it remains unclear which defenses to deploy at each stage and how to combine them. Prior empirical studies, fragmented by inconsistent attack-success-rate definitions and experimental settings, have evaluated defenses largely in isolation. Here we present the first systematic study, to our knowledge, of defense combinations both within and across pipeline stages, under a consistent threat model of direct, black-box, single-turn attacks. Our decision framework standardizes evaluation through a principled attack-success-rate formulation with controlled query budgets, together with explicit fairness rules. Across 19 attacks and 15 defenses, we find that no single defense is universally best, but well-chosen combinations achieve substantial safety with minimal utility degradation, yielding practical recommendations for layered defense pipelines.
Text extracted automatically; images, tables and formatting may be missing. Original: https://arxiv.org/abs/2609.21793