Multi-layer Indexer optimizes DeepSeek sparse attention
Original title: Multi-layer Indexer: Overlapping Selection and Attention in DeepSeek Sparse Attention
Multi-layer Indexer speeds DeepSeek-V3.2 decode up to 14.5% without retraining.
The paper describes Multi-layer Indexer (MLI) for sparse attention in long-context LLMs. Adjacent layers share indexer selections so indexers run concurrently with attention. Combined with IndexCache it reduces decode time while keeping accuracy close to the original model on DeepSeek-V3.2 and GLM-5.
- up to 14.5% faster on DeepSeek-V3.2
- no retraining or new weights
- accuracy within few points
- AI models
- DeepSeek-V3.2GLM-5
From the original article
1 min read at huggingface.co
Sparse attention cuts the cost of long-context inference by making the attention matrix sparse: each query attends only a subset of the sequence, chosen by a fixed pattern or, in recent production LLMs, learned per query. In DeepSeek Sparse Attention (DSA), the attention of DeepSeek-V3.
By Serdarcan Dilbaz, Arjun Krishna, Kiran Kamble · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.