AutoTavern agents build a Hearthstone AI that rivals top ladder players
Original title: Build, Don't Play: Self-Improving LLM Research Agents Develop a Hearthstone Battlegrounds AI That Rivals Top Players
In 20 days the system produced TavernGym and 23 TavernAgent versions whose last 20 games averaged 1.60 placement.
AutoTavern is a self-improving research system in which specialised LLM agents share typed memory, pass claims through verification gates, and use live Hearthstone Battlegrounds games to improve a simulator, opponents, policy and their own process. A human sets goals and rules for ambiguous mechanics. TavernAgent, trained with PPO, plays complete live-ladder games alone; its rating rose from 6,000 to about 15,000. On TavernBench it answers 91% of play tasks versus at most 63% for frontier LLMs. The authors say they will open-source TavernBench and TavernGym.
Unconfirmed: Leaderboard and timing comparisons are the authors’ reported results; the promised open-source release is not confirmed here.
- The public leaderboard measure shows 80% first places versus 59% for the server’s top ten.
- Those ten players average placement 2.34, against TavernAgent’s 1.60 over its last 20 records.
- TavernBench has 960 tasks in 40 scenarios.
- TavernAgent decides in 0.03 seconds; frontier LLMs need up to 26 seconds.
- Within a live turn’s one second it keeps 90% accuracy, and no LLM exceeds 26%.
- Organizations
- MCG-NJU
- AI models
- TavernAgent
From the original article
1 min read at huggingface.co
Can LLM agents carry a hard real-world problem from raw data to a system that holds its own against people? We study this in Hearthstone Battlegrounds, an eight-player auto-battler with stochastic shops and battles, hidden information and nearly a thousand interacting cards.
By Changlian Ma, Limin Wang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.