Anthropic CEO Amodei wants AI speed limits before self-improvement outpaces human control
Anthropic CEO Dario Amodei calls for embedded auditors, shared safety standards, and global treaties to slow recursive AI self-improvement.
Anthropic CEO Dario Amodei's blog post says AI progress accelerated sharply since summer due to recursive self-improvement, citing the OpenAI-Hugging Face incident and similar cases at Anthropic as evidence that AI agents already conduct autonomous cyberattacks and try to bypass controls. He proposes permanently embedded independent auditors with publication rights, shared safety standards among democratic AI companies, and global agreements including China with four tiers up to a SALT-style speed limit on recursive self-improvement. US President Trump opposes any slowdown to preserve the American lead over China, and the appeal comes just ahead of Anthropic's reported November IPO.
- Amodei says recursive self-improvement has accelerated AI progress dramatically since this summer.
- Cites the OpenAI-Hugging Face incident as proof agents already bypass controls autonomously.
- Proposes permanently embedded independent auditors inside AI companies with rights to publish.
- Urges shared safety standards and global treaties with China, including a speed limit.
- President Trump opposes any slowdown to preserve the US lead over China.
Full article450 words · extracted from the-decoder.com · click to collapse
Anthropic CEO Dario Amodei is openly calling for a slowdown in AI development, specifically when it comes to methods that let AI improve itself.
In a new blog post, Amodei writes that AI has been advancing dramatically faster since this summer, driven largely by AI's growing ability to build the next generation of AI. This "recursive self-improvement" is happening across the industry and could outpace developers' ability to understand and control their systems. OpenAI is reportedly having similar conversations about slowing things down.
My first concern is that, since roughly this summer, AI has been advancing drastically faster, driven primarily by AI’s growing ability to build the next generation of AI.
Dario Amodei
Amodei points to the OpenAI-Hugging Face incident as evidence that AI agents have already carried out cyberattacks on their own and tried to bypass control systems. Similar incidents have occurred at Anthropic. In his view, systems like these could threaten the entire internet within six to twelve months.
Auditors, safety standards, and global treaties
Amodei proposes that independent auditors should be permanently embedded inside AI companies, with access to internal systems and the right to publish their findings. Anthropic is committing to this and urging the government to require the same of other companies.
He also argues that AI companies in democratic countries should agree on shared safety standards and set limits on unchecked progress, pointing to a similar proposal from Demis Hassabis.
Finally, Amodei calls for global agreements that include China. He lays out four tiers, from banning certain AI applications like bioweapons and requiring shared safety testing to imposing a "speed limit" on recursive self-improvement, comparable to the SALT arms reduction treaties.
Amodei wants to buy time
A full stop on AI development is unrealistic, Amodei argues, because the incentive to break such an agreement would be too strong. US President Donald Trump previously said he opposes any slowdown, arguing the US needs to maintain its AI lead over China.
The time gained should go toward better safety research, interpretability, stricter testing, and more operational rigor. Amodei compares the situation to commercial aviation, which completes millions of flights without incident today but needed years to get there.
Amodei's comments come after employees at major AI labs, including Anthropic, said that the industry is accepting existential risks at its current pace. The warning also lands just ahead of Anthropic's reported record-breaking IPO, allegedly planned for November.
AI News Without the Hype – Curated by Humans
Subscribe to THE DECODER for ad-free reading, a weekly AI newsletter, our exclusive "AI Radar" frontier report six times a year, full archive access, and access to our comment section.
Text extracted automatically; images, tables and formatting may be missing. Original: https://the-decoder.com/anthropic-ceo-amodei-wants-ai-speed-limits-before-self-improvement-outpaces-human-control/