What Breaks Under Pruning in Smart Homes, and When? Evaluating LLM Degradation Across Architectures and Task Complexity
Pruning study across four LLM architectures finds dense models degrade sharply on smart-home tool calling while MoE models tolerate far more.
Researchers systematically study pruning-induced degradation in smart-home tool calling across four LLMs spanning dense Transformer, dense hybrid, and mixture-of-experts architectures, combining depth, width, hybrid, and expert pruning methods, and evaluate over 19,500 instances from three datasets after post-pruning supervised fine-tuning. Dense models show narrow safe pruning regions followed by sharp degradation, while MoE models tolerate substantially more pruning. Pruning degrades grounded specificity (operation, device, argument, value) before schema-level intent, and aggressive dense pruning can induce systematic over-refusal.
- Dense LLMs have narrow safe pruning regions; MoE models tolerate much more
- Grounded specificity (arguments, values) degrades before schema-level intent
- Aggressive dense pruning induces systematic over-refusal in tool calling
From the original article
1 min read at arxiv.orgPruning can reduce the deployment cost of large language models (LLMs), but its impact on context-grounded tool calling remains poorly understood. We systematically study pruning-induced degradation in smart-home tool calling across four LLMs spanning dense Transformer, dense hybrid, and mixture-of-experts (MoE) architectures, together with depth, width, hybrid, and expert pruning methods.
By Congjing Zhang · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.