SqueezeBench: Saturation-Resilient Evaluation of Mathematical Capability
SqueezeBench forces math submissions to strictly improve certified bounds instead of saturating fixed problems.
SqueezeBench is a saturation-resilient mathematics benchmark in which each problem keeps lower and upper records for an integer function f(n). A submission must supply a computable Lean function and a machine-checked proof that it bounds f, is nowhere worse than the current record, and is strictly better at some n. Ten extremal-combinatorics problems are released with 20 initial records. Frontier-model agents had 82 submissions accepted and improved every record; agents using two newer models then had 60 more acceptances and improved 19 of 20 again, while eight problems remain asymptotically behind the best published bounds.
- A success raises the certified bound instead of retiring the problem.
- Each record is a Lean function checked by two independent kernels.
- Checks reject functions that compute f by exhaustive search.
- Frontier agents had 82 acceptances and improved all 20 records.
- Newer models improved 19 of 20; eight problems still lag published bounds.
From the original article
1 min read at arxiv.orgAI systems are making rapid progress in mathematics, but this progress is hard to measure. Benchmarks of solved problems saturate, and in benchmarks of open problems each problem can be resolved for the first time only once. We present SqueezeBench, a benchmark that is resilient to saturation: a success raises the bar instead of removing the problem.
By Pablo Manrique · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.