DECE arena compares LLM agents across ten Eurogames
Original title: A Unified Arena for LLM Agents across Diverse Eurogame
Twelve model-harness setups played 480 three-player matches; 1,032 matches were completed in total.
Researchers introduce DECE, a unified arena for evaluating large language model agents across ten Eurogames that share interfaces and evaluation protocols. The games emphasize lasting strategic commitments, competition over shared resources, and multiple routes to victory. The primary study covers 12 model-harness configurations, nine models and four execution harnesses, in 480 three-player matches; main and supplementary studies total 1,032 completed matches. Rankings largely agreed across games, but strengths were game-specific, the same model varied by harness, and stronger play did not consistently mean better token or cost efficiency.
Unconfirmed: Individual model names and per-game scores are not stated in the abstract.
- DECE spans ten Eurogames with shared interfaces and protocols.
- Nine models and four execution harnesses were compared.
- Matched studies found harness choice changed a model's results.
- Competitive gains did not reliably improve token or cost efficiency.
From the original article
1 min read at arxiv.orgInteractive environments offer a practical way to evaluate large language model (LLM) agents, and competitive games extend this evaluation to decisions shaped by competing goals and other agents' choices. Yet existing evaluations confined to individual games or execution frameworks leave unclear which competitive advantages persist across settings.
By Fengrui Zhang · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.