SWE-Game: Can Coding Agents Build the Games We Want?
SWE-Game finds coding agents, led by Opus5, still score below 60 building games.
SWE-Game benchmarks coding agents on 247 tasks drawn from 41 executable Godot reference games spanning 13 gameplay categories in 2D and 3D. Five task types cover building from a brief or design document, completing a skeleton, repairing 83 injected faults, and porting from Godot to Unity. Across six models, Opus5 scores highest on every task type, but the best construction scores remain below 60, with Brief-to-Game at 50.38. Executable checks reach 92.59% balanced accuracy on human-labeled behaviors, ahead of a 78.41% video-based VLM judge.
- SWE-Game has 247 tasks from 41 Godot games across 13 gameplay categories.
- Tasks cover briefs, design documents, skeleton completion, 83 fault repairs, and Unity ports.
- Opus5 leads all five task types, but construction scores stay below 60.
- Executable checks reach 92.59% balanced accuracy versus 78.41% for a video VLM judge.
- Requirement omissions and gameplay logic errors dominate reviewed failures.
- AI models
- Opus5
From the original article
1 min read at huggingface.co
We introduce SWE-Game, a benchmark of 247 tasks grounded in 41 executable reference Godot games spanning 13 gameplay categories in 2D and 3D. Five task types cover development from a brief, implementation from a game design document, skeleton completion, repair of 83 injected-fault cases, and Godot-to-Unity porting.
By Xiaoyu Chen, Lai Wei, Jin Wang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.