Game2Real-Bench links toy-game scores to real visual reasoning
Original title: Game2Real-Bench: Do Toy Games Reveal and Transfer to Real-World Visual Reasoning?
Across 25 vision-language models, game and real scores correlate at 0.935; constraint reasoning rose in one-shot transfer.
Game2Real-Bench compares procedurally generated game tasks with real-world visual reasoning across perception, spatial reasoning, constraints, and sequential planning. It uses 1,000 verified game samples and 400 real samples aligned by shared reasoning requirements. Evaluations of 25 vision-language models found strong game-to-real consistency. One-shot game demonstrations improved constraint reasoning for several of four models but reduced fine-grained perception.
Unconfirmed: Transfer effects are reported for four models; the paper does not name them.
- Pearson and Spearman correlations between game and real scores were 0.935.
- Game2Real-Dataset contains nearly 24K verified samples across 18 tasks and 65 QA families.
- One-shot transfer was studied on four models, not the full set of 25.
- Real samples were aligned by reasoning requirements rather than visual similarity.
- Products
- Game2Real-BenchGame2Real-Dataset
From the original article
1 min read at huggingface.co
Visual reasoning is an important capability for vision-language models (VLMs) to understand visual evidence, spatial structure, constraints, and state changes. Existing evaluations mainly rely on real-world data or procedurally generated tasks. Real-world data preserve natural complexity, while procedural games provide controllable generation and verifiable solutions.
By Qingan Liu, Juntong Feng, Yue Zhu · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.