Google launches Gemini 4 Carbon first for cyber defenders
DeepMind claims 88.2% on SEC-bench Pro; Artificial Analysis ranks Carbon first on Intelligence Index v5 at 59.
3
Stories per day · 30 days
7 stories, newest first · one card per story
Stories tagged #leaderboards
Paper treats benchmarks as difficulty measures and proposes BenchFill
Authors diagnose mismatch and coverage gaps on 25 benchmarks across seven domains, then rewrite items to fill gaps.
Dating the Model: Hidden Dates in System Prompts Affect LLM Evaluation
Hidden dates injected into system prompts shift LLM scores by up to 14% and reorder leaderboards.
Coding Agents Have Converged: Why the SWE-bench Leaderboard Can No Longer Order Its Top Entries, and What to Measure Instead
Audit of 254 SWE-bench submissions finds top coding-agent entries statistically inseparable, so small leaderboard gaps no longer establish rank.
Why User Studies and Participant Experience Reporting Matter for VR Motion Privacy?
User studies explain VR motion privacy mechanism acceptance far better than physical deviation metrics, a study of three mechanisms finds.
Benchmark Radar: A Living Database and Search Engine for AI Benchmarks and Evaluation
Benchmark Radar provides a living searchable database of 1,283 AI benchmark records and 12,916 score observations drawn from 37 daily discovery sources.
What Does an LLM-Agent Leaderboard Rank Actually Compare?
A methodological study shows close LLM-agent leaderboard rank gaps on SWE-bench and similar benchmarks often do not support superiority claims.