What Wins a Vote? Formatting, Length, and Lexical Diversity in the French Compar:IA LLM Arena
French Compar:IA votes show bold text and vocabulary variety shift LLM arena rankings.
Researchers analyze 137,293 decisive French-language votes from the July 2026 Compar:IA LLM arena, including formatting analysis of 137,113 battles across 116 models. Length, bold text, and lists tend to co-occur, but bold usage and moving-average type-token ratio remain the most stable associations, at +11.0% and +16.8% win odds per standard deviation in the joint model. Adjusting for presentation shifts 36 of 116 models by at least ten ranks, yet the adjusted ranking does not clearly match external capability benchmarks better than the raw ranking. The authors recommend publishing both rankings as a sensitivity analysis.
- Study covers 137,293 French Compar:IA votes and 116 models.
- Bold text raises win odds about 11% per standard deviation.
- Vocabulary variety (MATTR) raises win odds about 16.8% per standard deviation.
- Full adjustment moves 36 of 116 models by at least ten ranks.
- Adjusted ranks do not clearly track external capability benchmarks better.
From the original article
1 min read at arxiv.orgLLM arenas turn pairwise human preferences into model rankings. Those preferences may reflect how an answer is presented as well as what it says. We take a stylometric approach to 137,293 decisive French-language votes from the July 2026 Compar:IA release; the primary formatting analysis includes 137,113 battles across 116 models, and the joint estimates use the 127,092 battles with all required measurements.
By Simonas Zilinskas · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.