OpenAI wants to consult elite mathematicians about how to not fumble again

After turning a string of spectacular mathematical results into a reputational crisis, OpenAI is consulting human mathematicians to help it figure out a less disastrous path forward.

On Monday, the company announced a new independent panel of mathematicians tasked with advising it and other AI companies on their interactions with mathematical research and the wider mathematics community, including how new results are presented and released. Its abrupt arrival caught many mathematicians by surprise. Researchers told The Verge the group is a good first step, but many said they were left with basic questions about what it will actually do, how much influence it will have, and whether OpenAI will actually listen to it, as well as concerns over whether such a small group of prominent researchers can represent the wider mathematical community.

The nine-member group is stacked with who researchers described to The Verge as celebrities in the field, drawn from institutions including Stanford, Harvard, Oxford, and Cambridge, with multiple Fields Medals and MacArthur “genius grants” among them. It will be hosted by Princeton, New Jersey’s Institute for Advanced Study (IAS), one of the world’s most revered centers of mathematical research and the former intellectual home of figures including Albert Einstein, John von Neumann, and J. Robert Oppenheimer.

OpenAI said the group, called the Advisory Group on Mathematics and Artificial Intelligence (AGMAI), will operate independently, with the freedom to offer advice the company has not requested, publicly comment on its impact on mathematics, and make its advice public, though it did not provide details on what this may entail. It appears that the group will be granted early access to OpenAI’s research in order to fulfill its task. Members will not be paid by OpenAI and the group can change its membership as it sees fit, OpenAI said. “Its value depends on its members being able to exercise their own judgement and challenge ours,” the company said. “The group will advise on the review and communication of emerging results: they will help OpenAI assess their significance, advise on how to coordinate their dissemination, and advise on academic and professional standards of mathematical research. It will also advise on how our tools can support mathematical research and learning.”

For all the emphasis on independence, it remains unclear how much influence the group will actually have or why it was formed in the first place. Simon Machado, a researcher at ETH Zurich in Switzerland, said the whole thing seemed “shrouded in mystery” and he questioned how the nine members were selected. The group’s bare-bones website says it “came together after OpenAI approached some of its members about establishing an external advisory board,” but that, following discussions with the company, the mathematicians decided to launch an independent group instead and invite others to join. It’s not clear which members were initially approached by OpenAI or how the remaining members were selected.

In a blog post published Tuesday, advisory group member Martin Hairer elaborated on the group’s origins and the thinking behind it, responding to what he called a “torrent of misinformation” surrounding its formation. “It is obvious that AI has already had a profound impact on mathematical research and raises numerous questions of correct attribution of ideas, priority, human understanding of ideas, etc.,” he wrote, adding that there is no sense in the community “taking a hard ‘ostrich’ approach of simply ignoring the AI labs and refusing to talk to them on principle.”

While acknowledging that the group was formed in response to an approach from OpenAI, Hairer stressed that not all of its members had been contacted by the company and that the resulting organization is “genuinely independent of OpenAI and any other of the so-called ‘frontier’ AI labs.” Beyond “obvious confidentiality requirements,” he said members have not signed any restrictive agreements governing what they can say publicly, adding that they receive no financial compensation and are relying on the IAS for technical support.

“The other fundamental fact is that our overarching aim is to represent as well as we possibly can the interests of the mathematical community,” wrote Hairer, a professor of mathematics at Imperial College London and the Swiss Federal Institute of Technology in Lausanne (EPFL). While noting that the group is hardly a representative sample of the community, he said he is “absolutely certain that all nine of us take this extremely seriously and are acting in good faith.”

However the members were selected, the group currently comprises an elite cadre of academic mathematicians. The small size raises further questions about whether the group has the breadth of expertise required to evaluate everything coming its way, given the highly specialized skills and knowledge required to operate at the frontiers of mathematics.

Francesco Fournier-Facio, an incoming mathematics professor at Heriot-Watt University in Scotland, described himself as pretty “ambivalent” about the panel, saying that greater engagement with mathematicians “can only make things better.” But he questioned whether this group fully reflects the community whose work and livelihoods are being upended by OpenAI’s quest to rack up mathematical trophies. “It feels like an ivory tower,” he told The Verge.

Machado similarly questioned whether the group’s members fully understand, or can speak to, the concerns of the broader mathematical community when it comes to AI. “They’re amazing mathematicians, but I don’t know if they are the people I want to represent me in more political questions,” he said. For mathematicians in their and similarly elite positions, Machado said, “I think that the reality they are facing is very different from the reality that most mathematicians are facing right now.”

Whether the group can earn mathematicians’ trust will ultimately determine how effective it will be at containing the fallout from OpenAI’s recent parade of missteps in announcing results and, more importantly, soften the blow for whatever the company is planning to announce next. In his blog, Hairer said members would “be very naïve to believe that the AI labs won’t try to spin whatever we say in a way that suits their PR machine, which dwarfs anything we could possibly come up with.”

Nevertheless, Hairer argued it would be hypocritical not to engage now that OpenAI appears to be making an effort, particularly after so many criticized the company’s conduct and poor communications. The group has yet to settle on the advice it intends to provide, he said, but there is a clear problem it is responding to: “It is a fact that AI companies have in recent months been producing some high profile mathematical results and that their publication and dissemination has been falling far short of acceptable mathematical practice, whichever way that is defined.”

For now, Hairer said the group is gathering input through conversations with colleagues, a public feedback form, and discussion on Proofs and Prompts, a blog that has emerged as a key forum for mathematicians reckoning with AI’s impact on the field.

But the group’s biggest challenge likely lies ahead. When announcing the advisory group, OpenAI casually dropped the news that its new, unreleased model — the same that resolved the Navier-Stokes Millennium Prize problem — has now “resolved more than 100 long-standing open problems across most areas of mathematics.” It has previously said it is closing in on another Millennium Prize problem. On its website, the group said dealing with this logjam is its first priority: “We are currently facing the very specific challenge of advising OpenAI on how to coordinate the release of a large number of significant results in mathematics that they report have been produced by their internal model.”

In part, the new effort at engaging mathematicians is an acknowledgement of the monumental PR disaster its attempt was, in which the company found itself fending off accusations of scooping researchers, failing to credit the human work its models built upon, and even stealing from those using its models. Those controversies led many mathematicians to wonder if the company understood — or even respected — the discipline and community it had barreled into.

After the last few months and with so much having been written about what mathematicians want, Kevin Buzzard, a mathematics professor at Imperial College London, wondered why OpenAI felt like it needed such an elite body to understand what the community wants. “It wasn’t proofs of hard theorems, it was better understanding of our subject,” he said. “It’s not entirely clear to me that you need a committee of brilliant people to hammer the point home.”