PVBench scores physical plausibility in generated video
Original title: PVBench: Quantifying Physical Plausibility in Video Generation with Object-Level Physical Measurements
The benchmark uses 600 prompts and 46 dimensions, with object-level measurements instead of only visual-language judgments.
PVBench quantifies whether generated videos obey geometry, kinematics, dynamics, and conservation laws. It covers 46 dimensions in seven domains and 600 prompts, 30% of which involve embodied agents such as robot arms, humanoids, quadrupeds, and multi-agent systems. Specialized vision models supply measurements including 6D pose, metric scale, surface normals, kinematics, and conservation-law residuals. The authors compare automatic metrics with human judgments and evaluate mainstream open-source and closed-source models. Prompts and a calibration protocol are released.
Unconfirmed: The abstract says mainstream models were evaluated but does not name scores or which models.
- Thirty percent of prompts involve physical embodied agents.
- Metrics cover both text-to-video and image-to-video generation.
- Human evaluation is used to check agreement with automatic scores.
- Named model results are not specified in the paper abstract.
From the original article
1 min read at huggingface.co
Recent video generation models produce visually compelling outputs, yet their physical plausibility remains poorly measured. Physical plausibility asks whether generated objects obey geometry, kinematics, dynamics, and conservation laws. Existing benchmarks rely on coarse visual-language model judgments or target embodied world models rather than video generation itself.
By Kaiwei Chen, Moran Li, Zhenye Gan · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.