GameHorizon Suite: Multi-Horizon Data and Evaluation in Gameplay

GameHorizon pairs 5,000 hours of AAA gameplay data from 21 games with offline and online multi-horizon model tests, evaluating 47 models in over one million invocations.

GameHorizon Suite is a dataset and evaluation framework measuring AI gameplay ability across multiple time horizons. Its automated annotator produces multi-horizon language instructions, and GameHorizon-Data contains 5,000 hours of aligned video, actions, and instructions from 21 AAA games recorded by 100 expert players. GameHorizon-Bench adds reproducible offline tests using thousands of standardized questions across three tasks, plus stepwise online tests that check whether offline scores predict real play. Evaluating 47 models through more than one million invocations revealed a clear task-difficulty hierarchy and large capability gaps between models. The dataset, annotator, and benchmark are planned for release. The story was listed on Hugging Face daily papers on 2026-09-20 and on arXiv on 2026-09-21; the two sources agree, with the Hugging Face entry providing additional detail on the benchmark structure and findings.

• GameHorizon-Data contains 5,000 hours of aligned video, actions, and multi-horizon instructions from 21 AAA games recorded by 100 expert players.
• An automated annotator generates multi-horizon language instructions.
• GameHorizon-Bench combines reproducible offline questions — thousands of standardized questions across three tasks — with stepwise online tests that check whether offline scores predict real play.
• 47 models were evaluated through more than one million invocations, showing a clear task-difficulty hierarchy and large capability gaps.
• The dataset, annotator, and benchmark are planned for release.
• Listed on Hugging Face daily papers on 2026-09-20 and on arXiv on 2026-09-21.