OpenFrontiers benchmark tests LLM scientific rediscovery
Original title: OpenFrontiers: A Collection of Open Research Problems and a Benchmark for Scientific Rediscovery
The benchmark grades 200 math problems with 110 resolved in 2026 under closed-book, live-web and sandboxed conditions, revealing rare closed-book success.
OpenFrontiers collects 12,151 open research problems across disciplines and pairs them with a 200-problem math benchmark. Problems include Lean 4 statements and documented solutions. Seven LLM agents evaluated under standardized budgets show closed-book rediscovery remains rare while live-web performance largely reflects retrieval.
- 200 problems total: 110 resolved in 2026 and 90 still open.
- Closed-book rediscovery at most 7 of 40 resolved problems.
- Live-web success mostly retrieves published solutions.
- HQ-50 subset further filters for quality.
- Dataset and benchmark released on Hugging Face.
- Organizations
- huggingface
From the original article
1 min read at huggingface.co
Large language models (LLMs) have shown increasing promise in research-level problem solving, yet a comprehensive collection of such problems is still missing, and evaluating them on research-level problems remains challenging.
By Jiarui Yao, Ruida Wang, Tong Zhang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.