Pruning for Efficiency, Paying in Fairness: Demographic Disparities in Pruned Speech-LLMs
Pruning SLAM-ASR audio encoders widens demographic WER gaps that aggregate error rates hide.
The study measures how audio-encoder pruning of SLAM-ASR changes word error rate across demographic groups on Fair-Speech and Common Voice. Pruning does not hit groups equally, and the gap between best- and worst-performing groups grows; only the largest encoder initially conceals that gap in aggregate WER. LoRA adaptation improves every group but benefits already-strong groups more, while English, Danish, and Dutch accent gaps on Common Voice persist without clearly widening.
- Audio-encoder pruning of SLAM-ASR does not affect demographic groups equally.
- The best-to-worst group WER gap grows across three encoder scales.
- Only the largest model initially hides disparities behind aggregate WER.
- LoRA lowers every group's WER but widens gaps for some groups.
- On Common Voice, accent gaps persist but do not clearly widen.
- Products
- Common VoiceFair-Speech
- AI models
- SLAM-ASR
From the original article
1 min read at arxiv.orgSpeech-LLMs are expensive to run, making compression important for real-world deployment. However, compressed models are usually selected using aggregate word error rate (WER), which can hide how pruning affects different demographic groups. In this work, we systematically study the effect of audio encoder pruning on SLAM-ASR for different demographic groups.
By Ganesh Pavan Kartikeya Bharadwaj Kolluri · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.