GameHorizon Suite: Multi-Horizon Data and Evaluation in Gameplay
GameHorizon pairs 5,000 hours of AAA gameplay data with offline and online multi-horizon model tests.
GameHorizon is a data and evaluation suite for gameplay across short and long temporal horizons. Its annotator builds multi-horizon instructions, and GameHorizon-Data contains 5,000 hours of aligned video, actions, and instructions from 21 AAA games recorded by 100 expert players. GameHorizon-Bench adds reproducible offline questions and stepwise online tests. Evaluation of 47 models through more than one million invocations shows a clear task-difficulty hierarchy and large capability gaps. The dataset, annotator, and benchmark are planned for release.
- GameHorizon-Data holds 5,000 hours from 21 games by 100 expert players.
- An automated annotator produces multi-horizon language instructions.
- Offline tests use thousands of standardized questions across three tasks.
- Online stepwise tests check whether offline scores predict real play.
- Forty-seven models were evaluated with over one million invocations.
From the original article
1 min read at huggingface.co
Modern video games provide a measurable testbed for AI models, combining abilities of visual understanding, instruction decomposition, goal planning, and precise action control over multiple temporal horizons. Existing datasets and benchmarks, however, either cover a narrow range of games, lack language instructions, or rely on high-variance online rollouts.
By Yiran Wang, Xingyilang Yin, Junfu Pu · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.