APEX few-step streaming video drops the extra fake-score network
Original title: Few-Step Streaming Video Generation via Condition Shifting
Condition shifting estimates the generated velocity inside the generator and cuts peak GPU memory 4.4% and iteration time 9.2% versus DMD.
The paper extends APEX to streaming video so few-step generators estimate the generated distribution's velocity field by condition shifting, removing Distribution Matching Distillation's separate fake-score network. APEX keeps S2-Avatar's frozen reference teacher, Self-Replay Forcing, and perceptual regularization. On 180-second videos, a 70-case human study and a 320-case StreamAV-Bench comparison, the authors report slightly higher Progressive instruction fulfillment and visual aesthetics than S2-Avatar, with the strongest human preference on instruction following. Motion quality favors S2-Avatar, and audiovisual synchronization is mostly tied.
- Progressive instruction score is 3.856 versus 3.831 for S2-Avatar.
- Visual aesthetics are 0.641898 versus 0.632606 across both tracks.
- Human net preference is +30.0% on instruction following and +4.3% on consistency and video quality.
- Evaluation uses three backbone evaluations per block and a one-step spatial Refiner.
From the original article
1 min read at huggingface.co
Few-step sampling reduces the cost of streaming video generation. Distribution Matching Distillation (DMD) supervises few-step generators with a separate fake-score network, which adds training memory and computation and must track the evolving generator. We extend APEX to streaming video by using condition shifting to estimate the velocity field of the generated distribution within the generator.
By Deyuan Liu, Jungang Li, Kaiwen Zheng · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.