HLA-WM: Hybrid Linear Attention for Long-Horizon Video World Models
HLA-WM improves long-horizon video world-model consistency without training while cutting memory use.
HLA-WM is a training-free hybrid linear-attention method for long-horizon video world models that suffer long-range forgetting in Gated DeltaNet. It caches compact chunk-wise transition summaries, retrieves scene-relevant history with camera geometry, and recomposes query-specific recurrent states. On the 60-second SANA-WM-Bench it improves all six revisit-consistency and camera-control metrics, including a 0.74 dB PSNR gain and a 28.5% reduction in rotation error, while using 12 times less historical-state memory than full KV caching with at most a 1.6% throughput drop.
- Training-free hybrid of geometry retrieval and Gated DeltaNet states
- 0.74 dB PSNR gain and 28.5% lower rotation error
- Twelve-times less historical memory than full KV caching
- Revisit-consistency gains hold on MBench-A across 547 samples
From the original article
1 min read at huggingface.co
Long-horizon video world models require persistent memory to preserve scene consistency over extended rollouts. Softmax attention retains the full generation history through a growing KV cache, whereas recurrent linear attention compresses history into fixed-size states with substantially lower memory cost.
By Zhuokun Chen, Feng Chen, Xi Lin · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.