LLMs lag human researchers in research taste benchmark
Original title: IdeaTaste: Do LLMs Share the Research Taste of Human Researchers?
LLMs reach only 26 percent citation matching and 18 percent idea matching with 2026 human papers.
IdeaTaste benchmarks LLMs by comparing research proposals from a 2025 literature snapshot to actual 2026 publications. Experiments cover six language models and four research areas with more than 90,000 proposals. AI agents fall short of human communities in both citations and concrete approaches. Richer context or larger populations do not close the gap.
- 26 percent citation matching
- 18 percent idea matching
- Six models tested
- Four research areas
- 90,000 proposals
From the original article
1 min read at huggingface.co
Research taste, the judgment of what to read, which problems to pursue, and which methods to try, increasingly shapes the work of AI scientists, yet it remains far less understood than their ability to execute research. We introduce IdeaTaste, a benchmark that scores an LLM by how closely the research proposals it writes from a literature snapshot frozen at 2025 match the work the same communities published in 2026.
By Shanshan Zhong, Chen Xu, Jingjie Ning · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.