Benchmark Radar: A Living Database and Search Engine for AI Benchmarks and Evaluation
Benchmark Radar provides a living searchable database of 1,283 AI benchmark records and 12,916 score observations drawn from 37 daily discovery sources.
Benchmark Radar combines daily discovery of benchmark papers, repositories, datasets, and releases from 13 direct connectors and 24 first-party feeds into a searchable catalog with model card mentions and score histories. The catalog contains 1,283 source records drawn from 4 benchmark catalogs plus 12,916 numeric observations on 790 records. The release includes a web dashboard with leaderboard, Pareto frontier of score versus usage, saturation and trend views, daily feeds, a CLI, and reproducible analysis. The paper audits the full catalog and examines benchmark saturation and limits of score comparisons.
- Searchable catalog of 1,283 benchmark records with 12,916 numeric observations
- Daily discovery from 13 connectors and 24 first-party research feeds
- Dashboard offers leaderboards, Pareto frontier, saturation views, and a CLI
- Audits catalog quality and analyzes benchmark saturation and adoption trends
From the original article
1 min read at huggingface.co
Benchmark researchers and developers of large language models (LLMs) and other AI systems need to find relevant evaluations, locate their benchmark datasets and code, and understand the settings behind reported scores.
By Koutian Wu, Junjie Zhou, Ergan Shang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.