LEMS advances SVD compression for LLMs
Original title: Advancing SVD-based LLM Compression via Layer-Wise Error Model Search
LEMS and KFAC-SVD deliver up to 4.8 percentage point accuracy gains on Mistral, Qwen3 and Llama 3 models.
Layer-wise Error Modeling Search improves rank allocation in SVD-based LLM compression by using local and global error surrogates with propagation bias. KFAC-SVD mitigates rank collapse with token-wise statistics. LEMS outperforms existing strategies with zero-shot gains up to 4.8 points on models up to 70B parameters. KFAC-SVD improves average perplexity by 15 percent.
- Layer-wise error surrogate with ILP
- KFAC-SVD for decomposition
- Up to 4.8 p.p. zero-shot gains
- Models up to 70B
- Mistral, Qwen3, Llama 3 families
From the original article
1 min read at huggingface.co
Low-rank SVD-based compression offers a powerful strategy to reduce the computational costs of LLMs. However, existing methods face two key limitations: (i) global rank allocation, where uncalibrated error proxies fail to capture complex error propagation, and (ii) decomposition quality, where Fisher-based estimators suffer from severe rank collapse.
By Moritz Thoma, Maximilian Groezinger, Maximilian Forstenhäusler · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.