ROWBench: Do Video Models Render What the Program Specifies?
PROWBench checks whether video world models render program-specified events across 170 episodes.
PROWBench evaluates whether programmable video world models visually follow fine-grained, program-specified events rather than only visual quality or coarse physical adherence. It contains 170 programmatically built episodes and 600 proxy videos, with replayable world records of entity states and timestamped events, including events outside the camera view. Generated video is checked against observable consequences of program execution using entity-control and long-horizon memory tests plus two VLM metrics, Logic-Render Alignment and Interaction Success Rate. The suite covers first- and third-person views, with synchronized multi-view data for a subset.
- PROWBench has 170 programmatic episodes and 600 proxy videos.
- World records log entity states and timestamped events, including off-camera events.
- Metrics cover entity control, long-horizon memory, Logic-Render Alignment, and Interaction Success Rate.
- A subset of episodes includes synchronized first- and third-person multi-view observations.
From the original article
1 min read at huggingface.co
Programmable world models separate executable dynamics from visual generation, offering a promising foundation for next-generation game engines. However, their visual adherence to explicit rules and interactions remains insufficiently evaluated. Existing benchmarks assess visual quality, controllability, and instruction or physical adherence, but rarely test fidelity to fine-grained, program-specified world events.
By Zheng-Hui Huang, Guixu Lin, Yu-Ju Tsai · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.