Simulated Control Evaluations Predict AI Safeguard Performance
Original title: Simulated Control Evals
SCEs using language models to simulate environments predict safety with 3.6 percentage point RMSE, outperforming BashArena by explaining 78-85% of variance.
Control evaluations extrapolate safeguard effectiveness from purpose-built settings to deployment. Simulated control evaluations (SCEs) replace environments with language model simulations for tool responses. On LinuxArena with ground-truth execution, SCEs show lower errors than BashArena across monitors at a 5% audit budget. Strategies from SCEs improve monitor performance on held-out attacks.
- SCEs achieve RMSE of 3.6 pp vs 10.1 for BashArena
- Over tenfold lower KL divergence from execution
- Explain 78-85% of between-environment variance in attack success
- Sample efficient with fewer than 5% task draws
- Strategies discovered through simulation improve held-out monitors
From the original article
1 min read at arxiv.orgControl evaluations measure whether safeguards can prevent an AI model from causing undesired outcomes even when it is intentionally trying to do so. These evaluations typically use purpose-built control settings and extrapolate their results to a model's deployment. However, the effectiveness of safeguards in deployment can depend on features of environments and tasks that control settings do not represent.
By Jeffrey Olmo · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.