ReaDy uses HLA recurrent states to speed speculative decoding
Original title: ReaDy: Recurrent States as Dynamic Context for Speculative Drafting in Hybrid Linear Attention Large Language Models
On Qwen3.8-27B and one B200, ReaDy averages 3.16 times autoregressive speed and beats DSpark on every benchmark.
ReaDy is a speculative decoding method that retrieves position-specific context from hybrid linear attention recurrent states during drafting. Each draft position uses a learned query against the target's committed states, and a lightweight state reader folds that context into draft representations with residual updates. Across seven mathematics, code, and dialogue benchmarks, it reports an average 3.16 times speedup over autoregressive decoding with Qwen3.8-27B on one NVIDIA B200. Versus DSpark, throughput rises 7.97% and acceptance length 8.50% on average. Code is at github.com/StartLuxLabs/ReaDy.
Unconfirmed: The authors say it is the first such method to the best of their knowledge.
- Low-rank state reads and fused operators limit extra compute and I/O.
- A two-stage adaptation builds on DSpark's parallel backbone and sequential head.
- Ablations cover reader capacity, state sources, and residual placement.
- Vendors
- NVIDIA
- Products
- NVIDIA B200
- Organizations
- StartLuxLabs
- AI models
- Qwen3.8-27B
From the original article
1 min read at huggingface.co
Speculative decoding accelerates autoregressive generation by drafting multiple tokens per step, yet feature-based drafters rely on target hidden features whose contextual emphasis may not align with the needs of subsequent draft positions.
By Bohan Sun, Zunhai Su, Xialie Zhuang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.