PhyRubric scores physical commonsense errors in text-to-video models
Original title: PhyRubric: Rubric-Grounded Diagnostic Evaluation of Physical Commonsense in Text-to-Video Generation
A judge trained on 3,788 videos tracks human ratings more closely than other open judges and matches Gemini 4 Argon.
Researchers introduce PhyRubric, a rubric that scores text-to-video clips against caption-specific semantic and physical items using timestamped evidence. PhyRubric-Bench covers 34 categories in five physical domains, and PhyRubric-Train has 3,788 videos used to fine-tune the open-source PhyRubric-Judge. Across ten text-to-video models, the authors say physical commonsense is still far from solved.
Unconfirmed: Correlation with human judgments and the claim that commonsense remains unsolved are the authors' reported results.
- Rubrics are built per caption before any video is seen.
- PhyRubric-Bench spans 34 categories and five domains.
- PhyRubric-Train contains 3,788 videos.
- Ten text-to-video models were evaluated.
- PhyRubric-Judge is compared with Gemini 4 Argon.
- AI models
- PhyRubric-JudgeGemini 4 Argon
From the original article
1 min read at huggingface.co
As text-to-video (T2V) models are increasingly positioned as world simulators, it is essential to evaluate whether their videos conform to physical commonsense. However, existing benchmarks rely on annotators who rate each video with a single score based on intuition, yielding ratings that are subjective and do not tell which physical principle is violated.
By Huiyi Chen, Weitai Kang, Shicheng Fan · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.