Bad Likert Judge: A Novel Multi-Turn Technique to Jailbreak LLMs by Misusing Their Evaluation Capability
Unit 42 details the Bad Likert Judge multi-turn jailbreak that abuses LLMs' evaluation capability, raising attack success rates over 60% across six frontier models.
Palo Alto Networks Unit 42 describes the Bad Likert Judge technique, a multi-turn jailbreak that asks a target LLM to act as a Likert-scale judge scoring the harmfulness of example responses. The highest-rated example in each scale can carry harmful content, bypassing the model's internal guardrails. Testing across six state-of-the-art text-generation LLMs showed an average attack success rate increase of more than 60% versus plain attack prompts, with tested models anonymized. The technique targets edge cases rather than typical use, and the article positions the work as guidance for defenders on potential jailbreak risks.
- Abuses the LLM's learned concept of harmfulness by having it self-evaluate on a Likert scale.
- Average attack success rate increased more than 60% versus plain attack prompts.
- Tested across six state-of-the-art text-generation LLMs with anonymized results.
- Builds on prior multi-turn techniques like Crescendo, Deceptive Delight and many-shot attacks.
- Targets edge cases; most models remain safe when operated responsibly.
- Vendors
- Palo Alto NetworksUnit 42
- Products
- Bad Likert Judge
- AI models
- Anthropic
From the original article
13 min read at unit42.paloaltonetworks.com
This article presents what we are calling the “Bad Likert Judge” technique. Text-generation large language models (LLMs) have safety measures designed to prevent them from responding to requests with harmful and malicious responses. Research into methods that can bypass these guardrails, such as Bad Likert Judge, can help defenders prepare for potential attacks.
By Yongzhe Huang, Yang Ji, Wenjun Hu, Akshata Rao, Danny Tsechansky, Jay Chen · unit42.paloaltonetworks.com
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.