StreamEval benchmarks interactive streaming video generation
Original title: StreamEval: Evaluating Interactive Streaming Video Generation Models
Across 37 models and 300 cases, the best Following and Edit Success scores are 36.90, 25.93, and 15.00.
StreamEval is a benchmark for interactive streaming video generation covering avatar dialogue, avatar performance, world generation, instruction editing, and reference editing. It contains 300 scripted cases lasting 15 to 300 seconds, scored on task execution, state consistency, visual fidelity, and audiovisual coordination. The authors find that high visual quality and identity retention do not imply reliable request execution or sustained state consistency. A leaderboard is available at streameval.ai.
- Each case specifies initial assets, incremental multimodal requests, and preservation requirements.
- Ten task-specific metrics use a 0-100 scale.
- The best Following score among seven avatar-performance models is 36.90.
- Text-conditioned world generation Following peaks at 25.93.
- Edit Success among three reported reference-editing models peaks at 15.00.
From the original article
1 min read at huggingface.co
As streaming video generation advances, enabling real-time interaction with generated video is becoming increasingly important. We introduce StreamEval, a unified benchmark for interactive streaming video generation spanning five tasks: avatar dialogue, avatar performance, world generation, instruction editing, and reference editing.
By StreamEval Team · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.