Re-Search tests whether LLMs can rediscover published research ideas
Original title: Re-Search: Dissecting LLM Ideation Through a Two-Player Questioning Game
On 200 ICML 2026 papers, six LLMs make nearly all progress in the first 30 of 100 yes-or-no questions.
Re-Search adapts Twenty Questions to research ideation: given only a published paper's problem, a model must recover its core idea through yes-or-no questions and binary oracle feedback. The benchmark uses 200 ICML 2026 papers and separates ideation from execution. Across six recent LLMs, most gains occur in the first 30 of 100 questions, after which models plateau. Controlled tests indicate the bottleneck is generating good questions and ideas, not judging them when they are supplied.
- Nearly all progress occurs within the first 30 questions.
- Models recognize a good idea or question when it is offered.
- Failures split between over-exploiting one direction and over-exploring many.
- Authors suggest adding externally sourced ideas rather than only improving generation.
From the original article
1 min read at arxiv.orgResearch ideation is iterative: researchers refine their ideas step by step based on feedback. Yet existing evaluations obscure this search process, making it difficult to understand how LLMs explore the space of research ideas and what limits their progress.
By Muhan Gao · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.