Artificial Analysis index revision drops Kimi K3.1 Pro five spots
Original title: New Artificial Analysis index widens the gap between open and closed models
Private tests now make up 60 percent of the index, and the open-to-closed gap rose from 3 to 12 points.
Artificial Analysis released version 5 of its Intelligence Index. Kimi K3.1 Pro, fourth at launch, fell to ninth behind eight closed models, and Grok 4.7 lost 12 points. For the first time, 60 percent of the index uses private test sets meant to limit training on known questions. Artificial Analysis also revised ambiguous and flawed items in its AA-Omniscience test. Researcher Nathan Lambert says reinforcement-learning setups that resemble benchmarks also drive the shifts.
Unconfirmed: The ranking drop is presented as evidence that the prior index overstated some models.
- The stated gap between open models and the leaderboard top is now 12 points.
- Outsiders can no longer inspect errors in the private prompts.
- Artificial Analysis says it corrected flawed AA-Omniscience questions for the new index.
- Organizations
- Artificial Analysis
- AI models
- Kimi K3.1 ProGrok 4.7
From the original article
1 min read at the-decoder.com
Artificial Analysis overhauled its Intelligence Index, causing Kimi K3. 1 Pro to drop five spots in the rankings. The article New Artificial Analysis index widens the gap between open and closed models appeared first on The Decoder.
By Maximilian Schreiner · the-decoder.com
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.