More than 20 leading AI researchers warn that automated AI research poses extreme risks
Hinton, Bengio, Pachocki and other researchers warn automated AI research could trigger an intelligence explosion.
More than 20 AI researchers, including Geoffrey Hinton, Yoshua Bengio, and OpenAI research lead Jakub Pachocki, published a paper warning that automating AI research and development could trigger an intelligence explosion. They say AI systems already write most of the code at the companies building them and could automate the full R&D pipeline within years, compressing years of progress into months. The authors warn that society may not keep up, control of superhuman systems could be lost, and power balances among nations, companies, and governments could erode. They urge policymakers to gain much greater visibility into how AI research is being automated.
- Signatories include Geoffrey Hinton, Yoshua Bengio, and Jakub Pachocki.
- Authors say AI systems already write most code at frontier labs.
- They warn full automation of AI research could arrive within years.
- The paper urges policymakers to monitor automated AI R&D more closely.
Full article243 words · extracted from the-decoder.com · click to collapse
In case you didn't have enough, here's another AI warning: More than 20 AI researchers, including Geoffrey Hinton, Yoshua Bengio, and OpenAI research lead Jakub Pachocki, warn of a possible "intelligence explosion" driven by self-improving AI in a new research paper. "Although there remains much uncertainty, AI R&D automation might soon trigger one," they write.
According to the study, AI systems already write most of the code at the companies building them and could automate the entire AI R&D pipeline within years. Progress that normally takes years could then happen in months. The authors warn society might not keep up, control over superhuman AI could slip away, and power balances between nations, companies, and governments could erode. They urge policymakers to gain far more visibility into how AI research is being automated.
The paper joins a growing list of warnings. 42 leading mathematicians recently called for more attention to existential AI risks, several AI lab employees have voiced fears that AI could destroy humanity, and Pachocki himself said no lab has solved alignment well enough "to continue responsibly scaling at maximum speed for much longer." Some Anthropic employees are reportedly already scouting safe havens in case AI goes sideways.
AI News Without the Hype – Curated by Humans
Subscribe to THE DECODER for ad-free reading, a weekly AI newsletter, our exclusive "AI Radar" frontier report six times a year, full archive access, and access to our comment section.
Text extracted automatically; images, tables and formatting may be missing. Original: https://the-decoder.com/more-than-20-leading-ai-researchers-warn-that-automated-ai-research-poses-extreme-risks/