MatAnyone-E cuts 4K video-matting cost with sparse memory matching
Original title: MatAnyone-E: Sparse Memory Propagation for Efficient High-Resolution Video Matting
Authors report about 7.4x faster inference and peak GPU memory at 7.6% of MatAnyone 2 at 4K.
Researchers present MatAnyone-E, an efficient extension of MatAnyone 2 for high-resolution video matting. It first propagates at low resolution to find uncertain regions and relevant memory, then runs high-resolution matching only there and uses a lightweight decoder. The authors say quality stays comparable to MatAnyone 2 while estimated inference is 7.4x faster and peak GPU memory falls to 7.6% at 4K. On one NVIDIA H200 with compilation, they report 73.07 FPS at 1080p and 28.62 FPS at 4K.
Unconfirmed: Speed, memory, and quality figures are the authors' estimates, not independently verified.
- Low-resolution propagation selects uncertain regions and relevant memory candidates.
- High-resolution matching is limited to those regions; other results are reused.
- A lightweight decoder is trained to reproduce the original decoder's representations.
- Reported speed is 73.07 FPS at 1080p and 28.62 FPS at 4K on one H200.
- Vendors
- NVIDIA
- AI models
- MatAnyone-EMatAnyone 2
From the original article
1 min read at huggingface.co
High-resolution video matting requires preserving fine details, such as hair strands and soft transparency, while maintaining consistency across frames. Memory-based methods achieve strong matting quality by matching current-frame features with information stored from earlier frames, but this process becomes increasingly costly as resolution grows.
By Shuyun Wang, Shangchen Zhou, Peiqing Yang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.