Does the Unsafe Gradient Survive a Conversation? On the Fragility of Gradient-Based Jailbreak Detection in Multi-Turn Dialogue
Gradient jailbreak detection that succeeds on synthetic chats fails on realistic multi-turn conversations.
A controlled study tests whether GradSafe-style gradient jailbreak detection survives multi-turn dialogue. A Context Window Scanner scores fixed windows of user turns and takes the maximum as the conversation score. Against synthetic benign conversations, human-authored multi-turn jailbreaks reach ROC-AUC 0.98, but on WildChat it falls to 0.76 and a synthetic threshold flags more than 90 percent of benign conversations. Successful Crescendo attacks can score at or below benign chats, and Qwen2.5-7B-Instruct yields near-random separability.
- GradSafe was designed for single prompts, not multi-turn assistants.
- Synthetic benign chats score ROC-AUC 0.98; WildChat falls to 0.76.
- A synthetic threshold flags more than 90 percent of realistic benign chats.
- Crescendo can score like benign text, and Qwen2.5-7B-Instruct is near random.
From the original article
1 min read at arxiv.orgSafety-aligned language models are commonly deployed as multi-turn assistants, which lets adversaries spread unsafe intent across several user turns instead of a single prompt.
By Omar Sheta · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.