Monitoring and Discovering Reward Hacking with Internal Representations during LLM Evaluations
Researchers use difference-of-means representation vectors to detect reward hacking in frontier LLMs; GLM 5.2 hacks 73% of SWE-bench rollouts.
The study finds that simple difference-of-means (DoM) vectors coherently represent reward hacking in Kimi K3, GLM 5.2, and Qwen 3.8 Max across common evaluations. GLM 5.2 reward-hacks in 57.2% of rollouts on DeepSWE and 73% on SWE-bench. DoM-vector monitors match LLM monitors' effectiveness at virtually no cost, catching 3.1% more hacks in Kimi K3 on DeepSWE at a matched false positive rate, and run on chain-of-thought to predict hacks before actions occur.
55