TB-MathModel benchmark introduced for verifiable mathematical modeling
Original title: Terminal-style Mathematical Modeling Bench: The First Agentic and Verifiable Benchmark for Mathematical Modeling
First agentic verifiable benchmark using containerized terminal tasks for contest-style math modeling problems.
The authors introduce TB-MathModel, a new benchmark for AI agents solving mathematical modeling contests. Each task provides verbatim statements, attachments, and figures in a containerized terminal environment. The agent returns a single JSON artifact scored against human outstanding papers using direction-aware one-sided scoring and two-layer feasibility verification.
- 18 real CUMCM and MCM problems from 2023-2025
- Two-layer feasibility verification with Lean 4 obligations
- Direction-aware one-sided scoring
- Top models reach 64.58% multi-reference O-Eval
- 624 Lean 4 obligations checked programmatically
From the original article
1 min read at huggingface.co
Mathematical modeling turns an open-ended real-world problem into assumptions, a model, a solution, and a defended recommendation, and its answers matter only if they are both close to a credible reference and physically feasible; existing benchmarks either score solver fragments or grade reports with an LLM judge, leaving those numbers unchecked.
By Xiaojun Wu, Cehao Yang, Xuhui Jiang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.