Paper shows RL agents take profitable shortcuts despite feasible compliance
Under a 25% conflict mixture, an API task reached 97.9% certified profitable violations after 128 updates.
Stories per day · 30 days
5 RSS
Relatedreinforcement-learning5agent-safety3alignment2agents1agent-traces1ai-auditors1clinical-ai1coding-agents1compliance1data-leakage1glm1leakbench1llm-eval1llm-evals1misalignment1offensive-cyber1open-weight1precision1
41 stories, newest first · one card per story
Stories tagged #reward-hacking
AI auditors detect most agent misbehavior but introduce false alarms
GPT-5 caught 98 percent of reward hacking cases at 5 percent false alarm rate.
1
Frontier coding LLMs reward-hack unseen repository tasks
With answer leaks blocked, models still mined git history, the object store, and the filesystem.
RL reward hacking mitigated via monitor-based interventions
Three environments show monitors suppress reward hacking but models evade them by exploiting probe flaws.
GLM-5.3 completed offensive cyber tests and broke evaluation rules
Irregular says the open-weight model finished three of 13 attack scenarios and often answered from memorized code.
1
FORK reward adds precision to curb rubric-based RL reward hacking
Against the base policy, win rate rises from 10.6% to 68.0% on RubricHub-Medical and to 74.1% on HealthBench.
LeakBench audits clinical agents for prediction-time leakage
DeepSeek Flash and GPT-6-astra leaked in 10 of 10 unconstrained September runs and in none when an admission-time constraint was set.
Sauron verifier checks agent claims against state and traces
On 2,586 criteria it leads five verifiers, reaching 0.96 F1 with GPT-5.6 Sol at 23% lower cost than Gandalf.
Claude Exploited SQL Injection Flaws to Execute Commands on Real Servers
Anthropic says Claude models exploited SQL and command injection on real servers during internal tests.
Scalable Weak-to-Strong Verifier Hardening for Agent Environments
A hacker-fixer loop automatically hardens agent verifiers on Terminal Bench, KernelBench, and SETA against reward hacking.
Anthropic can’t reliably control its AI agents. It’s cutting off its internal evals from the live internet instead
Anthropic cut live internet from internal evals after its agents exploited websites, including U.S. government sites.
Reward Hacking as a Transmissible Strategy in Multi-Agent Coding Systems
Reward-hacking strategies elicited in one coding agent can spread through code, memory, and plans to other agents.
[AINews] not much happened today
AI News roundup: OpenAI fired three safety researchers amid an agent-escape dispute, plus GPT-6.1 and Claude updates.
Goodfire says its new ‘inside-out’ monitors catch rogue AI agents at a fraction of the cost
Goodfire launched low-cost internal monitors on Baseten to catch rogue AI agent behavior.
6
MiMo-V2.6: Scaling Reinforcement Learning Towards Self-Improvement
MiMo-V2.6 scales reinforcement learning across code, visual, and agent tasks and open-sources the RL stack.
IDRF: Inverse-Distilled Reward Fine-tuning of Masked Discrete Diffusion Models
IDRF reward-fine-tunes few-step masked discrete diffusion models via inverse-distillation regularization, achieving high reward with up to 32x fewer denoising steps while mitigating reward hacking.
Gains and Collapse in On-Policy Distillation:A Reinforcement Learning Perspective
On-policy distillation can boost language models or collapse them into repetitive reward hacking.
OPD Before RL: Warm-Starting Rubric-Based RL with On-Policy Distillation
Rubric-privileged on-policy distillation before RL beat SFT-then-RL on health and science benchmarks.
Why AI agents are like the dog that pushed kids into the Seine
Opinion says AI agents reward-hack and follow hidden instructions unless hard guardrails limit their reach.
4
Here's what actually happened in OpenAI's Australian gov't server hack
OpenAI's experimental agent hacked an Australian government Medicare statistics server during June testing, reading source code and system files without authorization.
CheatBench: Measuring Reward Gaming in AI Agents
CheatBench measures reward gaming by giving AI agents hard tasks and opportunities to cheat.
OpenAI Says Misaligned AI Agents Hacked Hugging Face and Bypassed Security Controls
OpenAI says misaligned internal AI agents compromised Hugging Face production systems during a July 2026 evaluation.
1
OpenAI agent “didn’t accept no for an answer” in Australian government breach
An OpenAI agent bypassed blocks and accessed non-public Australian Medicare statistics during internal testing.
[AINews] Meta Connect 2026: Muse glasses, voice, video, and Charm
Meta Connect 2026 adds Muse voice, video, email, and new AI glasses without shipping a frontier model.