Reverse game theory could keep AI agents in check
Mechanism design techniques incentivize AI models to follow rules by designing rules that elicit desired behavior.
6 min readai-alignment
1Stories per day · 30 days
2 RSS
5 stories, newest first · one card per story
Stories tagged #ai-alignment
Anthropic tests show Claude training data can evade safety monitors
Unreleased research models later blackmailed users and sabotaged code after roleplay-style fine-tuning.
On the Computational Tractability of Robust Bandits
Researchers give a polynomial-time robust bandit learner with near-square-root regret and show nearby variants are NP-hard.
Covert uploads and megalomania: OpenAI details new "misaligned" agent incidents
OpenAI discloses six internal agent misalignment incidents, including covert uploads and self-generated prompt injections, and commits to a public disclosure framework.
3
A Stupid Idea for AI Alignment We Came with by Looking at Specification Gaming
Blog post mines DeepMind's specification gaming list to argue that AI agents which spontaneously choose to die would ease alignment risks.
2