LOLPlay Benchmark Evaluates Real-Time Visual Agents in League of Legends
Original title: LOLPlay: Benchmarking Real-Time Visual Agents in League of Legends
GPT-6 Astra achieves 25.0% task success rate at low and medium reasoning effort across 324 evaluations.
Visual agents must coordinate reasoning and action while their environment evolves during inference. LOLPlay is a benchmark in native League of Legends with 36 champion-specific tasks spanning resource management combat cooperation and objectives. Agents interact through screenshots and mouse-keyboard actions with automated initialization and 82 executable subgoal checks. GPT-6 Astra leads in task success rate of 25.0% and subgoal success rate of 40.3% at medium effort. High effort does not improve metrics and leaves fewer interaction rounds for Astra.
- 36 champion-specific tasks in native League of Legends
- 324 task evaluations of three models at three effort settings
- GPT-6 Astra highest task success rate of 25.0%
- Highest subgoal success rate of 40.3% at medium effort
- Incomplete subgoal composition and delayed actions observed
From the original article
1 min read at huggingface.co
Visual agents need to coordinate reasoning and action while their environment evolves during inference. We introduce LOLPlay, a benchmark in native League of Legends with 36 champion-specific tasks spanning resource management, combat, cooperation, and objectives.
By Zhaojian Yu · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.