StreamErase: Towards Ultra-Fast Streaming Video Erasing
StreamErase enables real-time streaming video object removal at up to 26 FPS on one GPU.
StreamErase is a method for streaming video object removal that first adds mask input to a text-to-video model, then transfers that erasing ability from a bidirectional foundation model to a unidirectional streaming eraser. Sliding-window causal attention with attention sinks, self-rollout, and KV-cache rolling are used to limit error accumulation and keep compute constant over arbitrarily long videos. The authors report sub-second latency and up to 26 frames per second on a single GPU, and they introduce the LongErase benchmark for long-video object removal.
- StreamErase distills a bidirectional text-to-video eraser into a causal streaming model.
- It reports sub-second latency and up to 26 FPS on one GPU.
- Sliding-window causal attention, attention sinks, and KV-cache rolling keep cost constant.
- A new LongErase benchmark evaluates long and potentially infinite video object removal.
- Products
- LongErase
- AI models
- StreamErase
From the original article
1 min read at huggingface.co
Current video object removal methods suffer from prohibitive latency (minutes per video) and non-causal processing that prevents real-time interaction. We present StreamErase, enabling sub-second latency with up to 26 FPS streaming generation on a single GPU.
By Zhuobai Dong, Xiaobing Tu, Xiantao Zhang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.