Code2Video-Compass benchmarks code-based video generation
Original title: Code2Video-Compass: A Comprehensive Benchmark for General-Purpose Code-Based Video Generation across T2V, I2V, and V2V
The suite has 2,104 tasks across T2V, I2V, and V2V, and quality falls as complexity rises.
Code2Video-Compass tests whether multimodal models can write programs that produce complete videos from text, image, or video inputs. It contains 2,104 tasks in 10 categories and 24 subtypes, scored on construction, animation, and state-dependency complexity. Evaluation uses 12 objective metrics and five subjective metrics from an MLLM judge. Tests of 15 models from nine families find leading systems still struggle as complexity increases.
Unconfirmed: Benchmark outcomes are the authors' reported results.
- Tasks draw on instructions, visual references, and complete videos.
- Objective scores cover operation, quality, alignment, and reference fidelity.
- An MLLM judge scores instruction following and holistic quality.
- Individual model names are not listed in the paper summary.
From the original article
1 min read at huggingface.co
Code-based video generation offers fine-grained control, targeted editing, and component reuse compared with direct video generation. This approach requires multimodal large language models (MLLMs) to translate visual and temporal requirements into executable programs that produce complete videos.
By Zhenghan Chen, Zekai Shao, Xin Lin · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.