Gradient-aware allocation and sliding windows for low-bit GDN states
Original title: Low-Bit GDN Recurrent-State Quantization: Global Gradient-aware Allocation and Pure-Write Sliding Windows
On Qwen3.5-4B and 9B, GGA plus PW-S8 beats STEPQuant@4 by 38.39 and 41.21 points on seven-task averages.
The paper addresses error from low-bit recurrent states in linear-attention GDN models, where quantized writeback can degrade long reasoning and lengthen outputs. Global Gradient-aware Allocation assigns head bit widths using quantization residuals and gradient sensitivity under a mixed-precision budget. Pure-Write Sliding Window keeps recent writes as FP8 pairs and merges them into the low-bit state only after they leave the window. On Qwen3.5-4B and 9B, both methods raised scores and shortened generations versus STEPQuant.
- GGA alone beat STEPQuant@4 by 19.48 and 29.84 percentage points.
- GGA plus PW-S8 reached seven-task averages of 69.83% and 78.63%.
- At @6, GGA cut mean generation length by 10.45% and 14.65% versus STEPQuant.
- Adding PW-S8 to GGA@4 cut mean length by a further 32.32% and 36.77%.
From the original article
1 min read at arxiv.orgLinear attention stores historical information in a recurrent state of fixed size, providing a compact memory representation for long-context inference. Reducing state precision further lowers per-request storage requirements, but quantized writeback errors enter subsequent recurrences, degrade prediction quality on long-reasoning tasks, and can be accompanied by repetitive generation and excessively long outputs.
By Qinghang Huang · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.