Quoting Terence Tao
Terence Tao warns AI agents racing to solve rumored open problems may incentivize researchers to hide directions, damaging open science.
Mathematician Terence Tao argues that fruitful open problems are being depleted non-renewably as AI-assisted effort floods any problem that gains attention. He observes that even rumors of someone working on a problem can trigger massive AI-powered effort to solve it before the original project matures. Tao warns this dynamic may push researchers to stop sharing promising directions, reversing centuries of open-science tradition.
- Tao describes open-problem supply as being mined non-renewably as AI scales research effort
- Rumor of work on a problem now triggers mass AI-driven attempts before the original project completes
- Potential outcome: researchers stop sharing directions, ending open-science norms
I wrote recently about how the collection of good, fruitful open problems is now being mined in a non-renewable fashion, leading to the potential scenario of these problems becoming scarce. [...] We have now seen that even the rumor of someone working on a problem can trigger a massive amount of AI-powered effort to flatten it before the original research project has time to reach its full potential. The incentives may now be pointing in the direction of no longer sharing any promising research directions with the broader community, which would reverse centuries of traditions of open science and do serious long-term damage to the future of the field. — Terence Tao Tags: ai-ethics ,…
This source does not provide full text. Read it at simonwillison.net.