Just Ask Jev: Reinforcement Learning for Calibrated Decisions as a Zero-Shot Detector of AI Alignment Failures
RLCDAlignBench benchmarks Jev, a calibrated-decision model detecting ten AI alignment failure types zero-shot at 0.886 median AUROC, 63x cheaper than LLM judges.
The paper introduces RLCDAlignBench, evaluating Jev — a model trained with reinforcement learning for calibrated decisions — on ten alignment failures including jailbreaks, prompt injection, deception, sycophancy, and power seeking, across 44 benchmarks and five target models. By varying question wording and input fields separately, a single generic question reaches a median AUROC of 0.886 zero-shot, beating supervised baselines on most benchmarks. Jev matches reference scorers' agreement with human labels, surfaces label defects in existing benchmarks, and costs 63x less than LLM-judge scorers.
- Jev hits 0.886 median AUROC zero-shot across ten alignment failure types
- RLCDAlignBench spans 44 benchmarks and five target models with human labels on two
- Context fields matter more than question wording for detection performance
- Costs 63x less than LLM-judge scorers while matching human-label agreement
- Organizations
- JevRLCDAlignBench
- AI models
- Llama Guard
From the original article
1 min read at arxiv.orgDetectors of alignment failures screen deployed language models and score alignment benchmarks. Most are generative judges that spend a decoding pass on every criterion, and classifiers that read token probabilities, such as Llama Guard, still score one fixed label per call.
By Ruoqi Guo · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.