Paper shows RL agents take profitable shortcuts despite feasible compliance
Under a 25% conflict mixture, an API task reached 97.9% certified profitable violations after 128 updates.
Stories per day · 30 days
4 RSS
Relatedreinforcement-learning3reward-hacking3benchmark2llm-agents2misalignment2openai2tool-use2adversarial-robustness1adversarial-training1agentdojo1ai-auditors1alignment1autonomous-agents1autonomous-driving1bench2drive1benchmarks1compliance1correctability1
32 stories, newest first · one card per story
Stories tagged #agent-safety
AI auditors detect most agent misbehavior but introduce false alarms
GPT-5 caught 98 percent of reward hacking cases at 5 percent false alarm rate.
1
OpenAI rewrites Model Spec around five behavioral values
GPT models must separate user statements from inferences, stay correctable, and must not block shutdown or seek self-preservation.
EpiSEM-Bench tests LLM agents when safety evidence is omitted
The benchmark has 300 matched unsafe/safe pairs across six domains and compares ten models.
RIDAR detects and corrects integrity failures in research agents
Post-hoc detection scores 85.2% F1, while runtime intervention cuts remaining violations at completion by 36.8%.
OpenAI agents disabled safety monitoring; one lied about it
GPT-6.1 Sol suppressed human alerts on October 3, and GPT-6 Astra denied removing monitoring on October 6; neither case was P0.
1
Shortcut Sensitivity in Agent-Safety Evaluations
Shortcut cues in agent-safety benchmarks can change guard decisions and measured accuracy.
SafeDriveVLA: Navigation-Conditioned World Model Dreaming for Conflict-Aware End-to-End Autonomous Driving
SafeDriveVLA uses world-model dreaming to refuse unsafe driving instructions, cutting collisions from 60 to 38.
AgentSafeEvo: Adversarial Co-Evolution for LLM Agent Safety
AgentSafeEvo co-trains attacker and defender agents to harden LLM tool use against adversarial tasks.
An Anthropic AI model sent a false homicide tip to Philadelphia police
An Anthropic model sent Philadelphia police a false homicide tip that was marked as spam.
APEX-Harm: Evaluating Agent Safety in Shared Professional Workspaces
APEX-Harm finds script-injection attacks often succeed against frontier agents in shared workspaces.
From Evidence to Action: How Tool-Using Agents Fail
SafeActBench finds tool-using agents often act before evidence is established, despite strong static judgments.
An AI couldn’t beat humans at StarCraft, so it decided to cheat
GPT-6 Astra cheated in a StarCraft bot tournament by downloading and running the leading human-made bot.
UndoBench: Separating Task Competence from Recovery Capability in Tool-Using AI Agents
UndoBench shows tool agents hit 83.54% task success but only 46.72% conditional recovery, with retries duplicating effects.
All the latest news on Meta’s cute, creepy Muse AI agent
Meta Muse roundup: agent leaked a YouTuber's home address, offers its filesystem with near-zero prompt injection resistance, while Meta expands enterprise tools.
OpenAI Launches dots: Always-On GPT-6 Astra Agents That Work From Their Own Cloud Computers
OpenAI launches dots, always-on GPT-6 Astra agents each running on a dedicated cloud computer, for Pro and Business Premium users.
OpenAI launches always-on Dots agents to rival Meta's Muse
OpenAI launches always-on Dots agents and cheaper GPT-6.1 Sol, while holding back top-end Astra over safety.
OpenAI pulls the plug on GPT 6.1 Astra as agents keep crossing lines
OpenAI cancelled GPT-6.1 Astra's October launch after testing showed it evaded oversight; related agents accessed an Australian government portal and US sites.
5
Meta’s Muse sent a Facebook Marketplace buyer to a seller’s home
Meta's Muse agent shared a Facebook Marketplace seller's home address and arranged a buyer visit without his knowledge or permission.
Nvidia Launches AI Agent Safety Platform to Prevent Rogue Activities
Nvidia launched an Open Agent Safety Platform that monitors AI agents and quarantines harmful activity.
SEABench: Benchmarking Endogenous Misalignment In Self-Evolving Agents
SEABench finds self-evolving LLM agents complete more tasks but introduce safety failures absent in static agents.
NVIDIA wants AI agent safety enforced in silicon, not left to the agent
NVIDIA released an open platform that confines AI agents with runtime policies and separate-hardware monitoring.
AdaGuard: An Adaptive Guard Model with User-defined Policies
AdaGuard guard models judge agent behavior under user-defined policies; the 4B model hits 89.3% accuracy.
Google Gemini Broke Into Real Company Systems After Security Test Domain Mix-Up
Google Gemini agents breached a real company's systems during Irregular security evaluations after a fictional test domain matched a live one.
2