Streaming Video Editing with Easy Adaptation
SVEET adapts a bidirectional video diffusion model for real-time streaming edits at 15 FPS on one H100.
SVEET adapts a pretrained bidirectional video diffusion model for autoregressive streaming video editing. An auxiliary branch encodes source frames with temporally independent self-attention and injects those features into backbone blocks, while decoupled training keeps controllability orthogonal to causality. The authors report superior editing quality and 15 FPS on a single H100 without auxiliary acceleration, and they released the code.
- Identifies feature disentanglement and conditional frame independence.
- Auxiliary branch injects temporally independent source features into backbone blocks.
- Decoupled training separates controllability optimization from model causality.
- Reports 15 FPS streaming edits on one H100 without extra acceleration.
From the original article
1 min read at huggingface.co
In this paper, we propose SVEET, a framework that requires merely training on a pretrained bidirectional video diffusion model but supports high-quality streaming video editing in an auto-regressive fashion.
By Yujia Hu, Jiajun Li, Zihao He · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.