Just Ask Jev: Reinforcement Learning for Calibrated Decisions as a Zero-Shot Detector of AI Alignment Failures
Jev detects ten alignment failures zero-shot, with a median AUROC of 0.886 on RLCDAlignBench.
Researchers introduce Jev, a model trained with reinforcement learning for calibrated decisions (RLCD), which answers many typed questions about one input and returns calibrated probabilities in a single call. RLCDAlignBench evaluates it on ten alignment failures, including sycophancy, jailbreaks, deception, prompt injection, hallucination, and power seeking, across 44 benchmarks and five target models. A single generic question reaches a median AUROC of 0.886 zero-shot, beats supervised baselines on most benchmarks, matches reference scorers' agreement with human labels, and costs 63 times less than LLM-judge scorers. Question wording matters little; performance depends more on input fields, especially those that encode the label.
- Jev scores many typed questions about one input with calibrated probabilities in one call.
- RLCDAlignBench covers ten failure types across 44 benchmarks and five target models.
- A generic question reaches median AUROC 0.886 zero-shot and beats most supervised baselines.
- Jev matches human-label agreement of reference scorers and costs 63x less than LLM judges.
- Input context, especially label-encoding fields, matters more than question wording.
- AI models
- JevLlama Guard
From the original article
1 min read at huggingface.co
Detectors of alignment failures screen deployed language models and score alignment benchmarks. Most are generative judges that spend a decoding pass on every criterion, and classifiers that read token probabilities, such as Llama Guard, still score one fixed label per call.
By Ruoqi Guo, Yi Liu, Gelei Deng · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.