Geoffrey Hinton: 'We're creating alien beings. I'm willing to choose humans'

Donald Trump is a “corrupt, senile felon” whose fixation on the term superintelligence proves he knows nothing about AI, said Geoffrey Hinton, “godfather” of the technology, in a wide-ranging interview where he doubled down on recent warnings that AI models pose an existential threat to humanity and scoffed at claims the industry could regulate itself.

The pioneering computer scientist, whose work helped lay the foundations of modern AI, has since become one of the technology’s most vocal critics. His latest remarks come after a tumultuous month during which a growing list of senior AI scientists and researchers have joined Hinton in ringing the alarm about how the technology he helped create could ultimately escape human control, sparking an industry crisis, the hasty assembly of congressional meetings and a rare global consensus that something must be done.

There has also been fierce pushback. Last week the leader of the world’s most valuable company singled out the Toronto-based Hinton for a scathing attack.

“I love [him]. I hate his predictions … All of his predictions have been wrong. Enough predictions,” Nvidia’s CEO, Jensen Huang, told the podcaster Ezra Klein. Huang, whose company makes the chips that AI firms require, dismissed Hinton’s vocal concerns that the pace and breadth of AI development at the “frontier” could soon outpace the human ability to control it, and particularly his warning that there is a 10-20% chance of AI causing human extinction in the coming decades.

“That [figure] is not grounded on science, is not grounded on research. Just because it comes from a scientist doesn’t make it scientific. Those predictions are hurtful,” Huang said. “Don’t think for a second just because you’re an alarmist that you’re doing a social good. It is not true. So I think that we ought to just all be wiser, more mature, be evidence-based, be scientific.”

But Hinton said those criticizing him are missing a broader point about the nature of risk. He is not specifically predicting anything, he said.

“I’m trying to convey that there is a really serious possibility of something happening. If, for example, you’re told there was a 10% possibility of a bus crashing, you wouldn’t get on the bus,” he said. “The point of giving numbers is to say this is a serious risk. We don’t know how serious, but it’s serious.”

Recent safety breaches have evoked comparisons to a techno-dystopia story. The recent unwanted hack of the tech company Hugging Face by a “swarm” of AI agents that escaped the boundaries put on them by their creator has divided the technology community, with some seeing it as an example of what could come, while others accuse critics of anthropomorphizing systems that are simply following instructions. Hinton thinks dismissing the scenarios is unwise.

“This used to be the domain of science fiction. You’re going to have AI agents and they’re gonna go rogue. Oh, come off it. And they’ll go rogue and they’re going to learn how to communicate with each other, even though we didn’t think there was a way. Oh, come off it!” said Hinton. “And when they figure this out, they’re going to have big emotional expressions like, ‘oh my God!’ Well, of course, they don’t really mean ‘oh my God’. They’re just parroting the words we use. They’re going to learn to sacrifice themselves for the common good!’

“All this stuff just a few years ago was written off as complete fantasy. And it’s all happened.”

He heartily concedes that neither the warnings of AI critics nor the promises of its boosters can rely on precise predictions of what the technology will do. But, he argues, uncertainty is not a reason for complacency. “The unique problem with AI is that if it goes wrong, if it takes over, you don’t get a second chance. We can’t say, ‘Well, we lost that time. Let’s rerun the game.’ It’s not like that.”

It’s worth noting that Hinton has a record of seeing things where others did not. He toiled in relative obscurity at the University of Toronto during two AI “winters” when his branch of the field – trying to teach machines to operate in a manner similar to the human brain – was considered obscure. He was fixated on the idea that, like the brain, computer systems could be designed to learn patterns from data, rather than to follow only explicit, prewritten rules.

“We believed that if you could figure out how the brain learned, machines learned the same way, and you had enough compute power, they would do amazing things,” he said.

But the nascent field of neural networks was mocked by his peers.

As recently as 2009, when Yoshua Bengio, Hinton’s Canadian colleague and close collaborator, sent a paper to a machine learning conference, one of the referees told them: “This paper has no place in a machine learning conference. It’s about neural networks.”

It took Canada’s government, which funded years of seemingly fruitless tinkering by Hinton and his small team (some of whom, like Bengio, are now dominant figures in the AI field), to realize the possibility of neural nets.

In 2012, he and his students developed an image recognition program that vastly outperformed rivals. That year, Google acquired Hinton’s company DNNresearch Inc for $44m; neural nets now underpin the models of the most valuable AI companies. Hinton joined Google to continue his research into deep learning, while also working at the University of Toronto. “In the long run, huge new breakthroughs come from basic research,” said Hinton.

The vindication sits on the mantel above the fireplace of Hinton’s red-brick home in a leafy Toronto neighbourhood. Among the jumble of trophies is the small silver dish of the Turing award – often called the Nobel prize of computing – which he won alongside Bengio and colleague Yann LeCun. (Hinton also has the Nobel prize in physics.)

But in 2023, Hinton had announced his resignation from the tech giant to be able to “freely speak out about the risks of AI”.

Today, his concerns extend to the powerful figures building some of the AI models, and the political beliefs of those men. Hinton said that the AI venture capitalists Marc Andreessen and Peter Thiel, and the CEO of the newly rebranded SpaceXSI, Elon Musk, “don’t believe in democracy”. They see themselves as “Übermenschen” who believe they should “control society” and that Trump is a useful instrument in that aim, he said.

The US president, for his part, no doubt noticing how the huge valuations of AI firms have propped up the stock market, has claimed the only thing needed to rein in AI is a “smart president”, rejected any government safeguards in favour of allowing companies to regulate themselves, and wrote on his social media page that anyone who doesn’t use his preferred new term for AI, “super intelligence”, is “THE ENEMY”.

Hinton called Trump “a corrupt, senile felon”.

“I’m sure the oil and asbestos companies would have liked to [regulate themselves] as well,” he said. “Do we think tobacco companies should have been allowed to design the warnings on the packets? So that they could print them in a six-point font, in dark grey on black?”

Trump’s use of “superintelligence” proves he understands “almost nothing” about AI, Hinton added.

“He’s taken a technical term which has a very clear meaning and then reused it,” Hinton said. “Superintelligence means smarter than people, and has been around for at least 20 years. It’s got a very clear meaning. And now he says we’re going to call AI ‘superintelligence’. You can’t make it smarter than people just by calling it smarter than people.”

The speed and scope of developments has promoted a worry of catastrophes and of society’s inability to grapple with the nature of the challenge. But Hinton also sees an additional obstacle: helping people understand the “alien beings” that the companies are creating.

“People have a very bad model of what people are, and that’s why they fail to understand how these AI models are very like us. Most people don’t understand what consciousness is. They don’t understand what subjective experience is, and they’re very confident that even though they don’t understand it, they’re sure AI doesn’t have it.”

He rejects the idea that the human mind possesses some mysterious quality that machines, by definition, cannot share.

“It’s obvious that these AI models understand what they’re saying. You couldn’t possibly answer all the questions without understanding the question. It’s ridiculous to say they don’t understand.”

But the scope of shift in societal thinking around these models is profound, said Hinton.

“There’ve been three big revolutions where people ceased to be so important. The first was Copernicus, where we weren’t at the centre of the universe any more. How long did it take people to adapt to that? It took the Catholic church nearly 360 years to apologize to Galileo. And Darwin? Well, there’s people in Kansas that still haven’t adapted to that one,” he said. “Now we’re creating alien beings – meaning we’re not the only intelligent beings around. And the reality is, we only have a couple of years to adapt to that.”

In recent months, the AI giant Anthropic, maker of the popular chatbot Claude, has held private meetings with religious scholars, theologians and philosophers to look into the possibility of teaching AI to be “good” and to grapple with the possibility that the models have consciousness. Anthropic’s CEO, Dario Amodei, said the conversations had drawn on centuries of religious and philosophical thought about virtue, compassion, honesty and moral character, with the aim of making ethical behaviour more reliable.

“Once you start talking about, ‘We have a moral obligation towards these alien beings,’ you lose your audience,” said Hinton. “I’ve struggled with this issue. We care about people. [But] we’re making these things and there’s nothing to stop us giving these things – which are more intelligent than us – less rights than us. I’m willing to be selfish and choose human beings. Of course, we should try to make them so that they’re very happy in a symbiotic relationship with us, where our concerns are weighed more heavily than their concerns.”

Hinton is not alone in arguing that the best route to avoid societal collapse is to develop genuinely benevolent models. He hopes that whichever country is the first to develop this capacity, it would tell allies and rivals alike, because “no country wants AI taking over” in another country. There is a “vested interest in sharing knowledge”, he said.

“The reality is, we’re making beings of a different kind. And the big tech companies keep saying: we want it smarter. This is crazy. We should care as much about all the other aspects of a being as you do about its intelligence.”

Much of that depends on the nature of these models, he said, and their capacity to understand how to develop good behaviour.

“When you train them on everything on the internet, they develop all sorts of horrible tendencies and then you get them to repress their horrible tendencies,” he said. But suppressing bad behaviour is not the same as creating a good character. “It seems like the way one would design a Victorian gentleman – but not something we’ll need to live with.”