Searching for "Harmful Refusal": A Psychometric Audit of an AI Safety Benchmark
A psychometric audit finds HarmBench and HELM Safety do not isolate a single harmful-refusal attribute.
The paper audits whether harmful refusal is a single measurable attribute in HELM Safety. Of four candidate datasets, three are saturated, leaving HarmBench. Multidimensional item-response theory indicates HarmBench does not measure one attribute, and differential item functioning finds items where same-ability models from different developers score differently; those flags largely disappear under scope-specific matching. The authors argue that averaging HarmBench with other datasets into one HELM score hides saturation and conflates distinct harm behaviors.
- Three of four candidate HELM Safety refusal datasets are saturated.
- Multidimensional item-response modeling rejects a single HarmBench attribute.
- Same-ability models from different developers differ on some items.
- Aggregate safety scores can hide saturation and conflate harm behaviors.
From the original article
1 min read at arxiv.orgSafety benchmarks typically report one overall score for a suite of datasets, each of which may target one or more safety-related attributes, so models with similar overall scores can have very different attribute profiles. Comparing models is more tractable at the level of individual attributes, yet it is often unclear whether even a single dataset's scores isolate any single attribute.
By Christopher M. Stewart · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.