VideoMaMa++ keeps video mattes consistent across chunks
Original title: VideoMaMa++: Temporally Consistent Video Matting via Preserve-and-Refine Token
Preserve and refine tokens let sliding-window inference carry mattes across chunk boundaries without extra training.
VideoMaMa++ is a mask-guided diffusion video-matting framework aimed at flicker and discontinuous transparency when short chunks are matted independently. Separate RGB and mask token streams interact through full spatio-temporal self-attention. Learnable preserve and refine tokens support sliding-window inference that propagates finished mattes across boundaries, with a shared reference anchor to limit error buildup on long sequences. The paper says it beats prior diffusion methods on standard benchmarks and in-the-wild long footage.
- Diffusion backbones cited are limited to chunks of 6 to 24 frames.
- No additional training is required for the cross-chunk inference scheme.
- A globally shared reference anchor is used at every chunk.
- The paper claims unique consistency across chunk boundaries on long videos.
From the original article
1 min read at huggingface.co
Diffusion-based video matting has recently emerged as a leading approach, leveraging video diffusion priors from internet-scale pretraining to achieve strong zero-shot generalization from limited synthetic data. However, the heavy compute of these diffusion backbones limits inference to a short chunk of just 6 to 24 frames, so even ordinary videos span multiple chunks.
By Sangbeom Lim, Seoung Wug Oh, Heeji Yoon · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.