HLA: Expressive Hybrid Linear Attention via Chunk-Wise Dynamic Mixing
Hybrid Linear Attention adds query-dependent chunk routing to Gated DeltaNet, lifting long-context scores on Qwen3.5.
Stories per day · 30 days
1 RSS
5 stories, newest first · one card per story
Stories tagged #hybrid-attention
Multilinguality in Hybrid Attention LLMs
Hybrid-attention LLMs learn multilingual alignment faster when a full-attention layer precedes recurrent layers.
dQwen3.5: Hybrid-Attention Diffusion Language Models
Researchers adapt Qwen3.5 hybrid-attention backbones (0.8B-9B) into diffusion language models, reaching training loss in half the tokens versus full attention.
Video DeltaNet: A Video-Native Hybrid Attention for Livestream Video Generation
Video DeltaNet combines softmax and linear attention for video generation, cutting DiT denoising of a 14.3-second 768p video to 6.7 seconds.
OpenVDN/vdn-minimax-h3 — new model trending #12 on Hugging Face
OpenVDN releases VDN-H3, an open hybrid-attention video model on MiniMax H3 that renders a 14.4-second 768p clip in 11.23 seconds on 8 B200 GPUs.
7