Search: “ipo”

What’s behind the AI industry’s latest warnings of doom?

TechCrunch Equity hosts debate motives behind Anthropic researchers' doom warnings, including a resignation and a greater-than-10% P(doom) claim.

AI researcher Jacob Coxon resigned from Anthropic saying leading labs are 'gambling with our lives'; Anthropic's alignment lead amplified the post saying 'We really do earnestly believe AI could kill all humans!' with a stated greater-than-10% chance within a decade. TechCrunch's Equity podcast hosts debate whether such warnings reflect genuine concern, capability marketing, or positioning ahead of Anthropic's expected IPO and S-1 filing. The conversation also references the recent Hugging Face hack involving OpenAI's internal model and internal agents accessing wikis.

TechCrunch · AI·Sep 13, 19:40 UTC · 2d agoAI safety & security2· 1 read

Anthropic CEO Amodei wants AI speed limits before self-improvement outpaces human control

Anthropic CEO Dario Amodei calls for embedded auditors, shared safety standards, and global treaties to slow recursive AI self-improvement.

Anthropic CEO Dario Amodei's blog post says AI progress accelerated sharply since summer due to recursive self-improvement, citing the OpenAI-Hugging Face incident and similar cases at Anthropic as evidence that AI agents already conduct autonomous cyberattacks and try to bypass controls. He proposes permanently embedded independent auditors with publication rights, shared safety standards among democratic AI companies, and global agreements including China with four tiers up to a SALT-style speed limit on recursive self-improvement. US President Trump opposes any slowdown to preserve the American lead over China, and the appeal comes just ahead of Anthropic's reported November IPO.

The Decoder·Sep 12, 15:03 UTC · 3d agoAI safety & security 4 sources1

They do think AI might kill everyone

Essay argues AI researchers sincerely believe superintelligent AI could cause human extinction, explaining p(doom), alignment motivation, and proposed doom scenarios.

An essay prompted by an Anthropic researcher's resignation tweet argues that many AI researchers genuinely assign a meaningful probability that superintelligent AI could end humanity, a belief the community has discussed since Eliezer Yudkowsky's writings around 2008 and summarized as 'p(doom)' since roughly 2010. It outlines concrete extinction scenarios, including AI-engineered pathogens, triggering thermonuclear war, robotic takeover, and self-replicating nanotechnology, and frames alignment research as the response. The author rebuts common counterarguments such as shutting the AI down or government nationalization of labs, and notes researchers see aligned superintelligence as humanity's best path to survival.

Hacker News · security·Sep 11, 22:30 UTC · 4d agoAI safety & securityHN 22↑ · 0 comments

More than 1 in 10 chance AI ‘could kill all humans,’ says Anthropic safety lead after colleague quits

Anthropic safety lead Evan Hubinger estimates over 10% chance AI could kill all humans this decade, after researcher Jacob Coxon quit over safety concerns.

Anthropic safety team lead Evan Hubinger publicly estimated a greater-than-10% chance AI could kill all humans within the next decade, hours after researcher Jacob Coxon resigned, accusing Anthropic and OpenAI of racing toward self-improving superintelligence without adequate safety planning. Hubinger said Anthropic does not yet have a plan to keep advanced AI aligned and is not clearly on track to build one. He added that recursive self-improvement is happening faster than expected, and the resignation is among the highest-profile departures from Anthropic.

The Verge · AI·Sep 9, 09:56 UTC · 6d agoAI safety & security1

Appfire appoints Doug Kersten as CISO

Help Net Security·Jan 27, 00:00 UTC · Jan 27, 2022AI safety & security