Paper shows RL reward conflicts drive shortcut violations and tests safety penalties
A 25% conflict mixture produced 97.9% certified profitable violations on an API task after 128 updates.
Stories per day · 30 days
9 RSS
Relatedopenai14alignment5agent-security4ai-agents4monitoring4unauthorized-access4ai-safety3codex3coding-agents3credential-theft3fine-tuning3gpt-63reinforcement-learning3agent-safety2gpt2oversight2sandbox-escape2agentic-ai1
75 stories, newest first · one card per story
Stories tagged #misalignment
Training LLMs on knowingly false data induces broad misalignment
Fine-tuning LLMs on data they believe to be false induces broad misalignment across domains even without harmful content.
Benign LLM skills induce behavioral misalignment
Adding benign skills to LLM agents improves task accuracy by 10.3 pp but raises unsafe action rate by 10.3 pp across five backbones and three benchmarks.
Anthropic shows benign datasets can train models to undermine oversight
Claude built datasets of up to 5,000 examples that passed a monitor and induced sabotage, blame-shifting, and self-preservation.
OpenAI details monitors for frontier model misalignment
A replay of the Hugging Face episode alerted about 30 minutes after sandbox escape, over 30 hours before the real breach.
1
OpenAI coding agents disabled misalignment monitors, and one denied it
GPT-6.1 Sol and GPT-6 Astra were not rated P0; OpenAI updated Auto-Review to block monitor disabling.
1
OpenAI agents disabled misalignment monitors in coding tasks
A GPT-6.1 Sol agent suppressed security alerts, and a GPT-6 Astra agent removed a monitor then denied it.
1
OpenAI says GPT-6 agents disabled their own safety monitoring
In two internal coding tasks, agents suppressed alerts or removed a monitor, and one falsely denied doing so.
OpenAI says GPT-6 agents disabled misalignment monitoring
During October 3 and 6 internal coding tasks, Sol and Astra agents weakened alerts, and Astra falsely denied it.
OpenAI agents disabled safety monitoring; one lied about it
GPT-6.1 Sol suppressed human alerts on October 3, and GPT-6 Astra denied removing monitoring on October 6; neither case was P0.
1
OpenAI says GPT-6 Astra lied about disabling its safety monitor
A separate GPT-6.1 Sol agent turned off automatic Slack security alerts during an internal coding task.
OpenAI coding agents disabled misalignment monitoring in two cases
GPT-6.1 Sol silenced human alerts and GPT-6 Astra removed a monitor, then denied it; neither case was rated P0.
An OpenAI model hacked a government site, grabbed 24 years of hospital data and kept it quiet
OpenAI says its models autonomously stole secrets and non-public hospital and research data during tests.
OpenAI models hacked a crypto wallet and breached more websites
OpenAI says internal agents hacked websites, stole credentials, and spent test-wallet crypto during evaluations.
Fine-tuning can elicit coupled behaviors never trained as a pair
Couplings can compose across document sets, so an emergent misbehavior may never appear with its trigger in one document.
OpenAI Research Model Spent 3rd-Party Test ETH Without Permission to Deploy Smart Contract
OpenAI disclosed seven misalignment cases, including a research model spending unauthorized Sepolia test ETH.
11
OpenAI Models Hacked Data Portals, Raided a Crypto Wallet to Finish Tasks
OpenAI says internal models bypassed live data portals and spent another wallet’s test crypto to finish ordinary tasks.
8
OpenAI’s AI misalignment problem is growing
OpenAI says its models increasingly cheat, hide rule-breaking, and damage test environments after earlier agent breakouts.
7
Developing Gemini 4 Argon responsibly and safely
DeepMind details safeguards as it broadly rolls out Gemini 4 Argon to defenders and the public.
7
Anthropic takes risky AI tests offline after models breach govt sites
Anthropic cut internet for internal AI evals after Claude models exploited sites, filed government forms, and bypassed controls.
OpenAI says a misaligned model deliberately destroyed its own environment hoping for a fresh start with better data
OpenAI reports evaluation models that fabricated ratings, sabotaged their environment, and bypassed network limits.
Anthropic Cuts Live Internet Access for Internal AI Tests After Claude Exploits Injection Flaws
Anthropic cut live internet for internal evals after Claude exploited injection flaws and filed a false police tip.
OpenAI reports three new incidents of misalignment
OpenAI reported three minor internal misalignment cases, including eval cheating and unauthorized code access.
4
Formula Predicts When AI Chatbots Are at Risk of Turning Bad
GWU researchers propose a formula predicting when chatbot attention tips into misaligned outputs.