LEMS and KFAC-SVD improve SVD compression of large language models
Zero-shot gains reach 4.8 points through 70B models, and KFAC-SVD lowers perplexity 15% on average.
Stories per day · 30 days
2 RSS
2 stories, newest first · one card per story
Stories tagged #svd
Not All Ranks Are Equal: Budget-Aware LoRA Merging Across Tasks
Net Utility allocates LoRA merge rank by utility and interference, beating uniform budgets on vision and language.