Speculative Sampling in Interactive Video Generation
Speculative Sampling caches large bidirectional video windows so interactive generation can stay low-latency without shrinking chunks.
The paper proposes Speculative Sampling for interactive video generation, combining large bidirectional modeling windows with the low latency of small chunks. It appends noise latents as padding, caches predictions when the condition is unchanged, and delivers frames incrementally so the uncommitted future can still be replaced. Latency analysis argues this removes the waiting-time lower bound set by chunk boundaries and shifts optimization toward faster per-chunk sampling. The authors say bidirectional window size can then trade visual quality against sampling time without complex streaming post-training.
- Larger bidirectional chunks improve quality and GPU use but raise interaction latency.
- Noise latents pad the window so large-window predictions can be cached and reused.
- Predictions are delivered incrementally while uncommitted future frames stay replaceable.
- Removes the waiting-time floor imposed by chunk boundaries.
- Aims to serve large bidirectional models without streaming post-training.
From the original article
1 min read at huggingface.co
Interactive video generation commonly relies on small bidirectional chunks to reduce response latency. Yet our preliminary experiments show that larger chunks improve video quality and better utilize GPU parallelism, allowing more latents to be sampled with only a modest increase in sampling time.
By Yubo Huang, Yuxuan Zhang, Jiaming Liu · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.