Do Audio Language Models Hear and Read Distinctive Features Alike?
Study finds most audio language models do not align heard and read features, except voicing in Qwen2.5-Omni.
Researchers test whether audio language models encode a phonological distinctive feature in the same direction when a phoneme is heard and when it is read. Across 6 models, 7 features, and 15 languages from 11 families, only voicing in the two Qwen2.5-Omni models exceeds a random-pairing reference after multiple-testing correction. In three of six models, voicing keeps one audio direction across 14 languages with enough minimal pairs. Model family, not size, predicts which stream represents a feature, and the random reference varies by a factor of seven between models.
- Six models, seven features, and 15 languages were compared.
- Only Qwen2.5-Omni voicing beat the random-pair reference after correction.
- Model family, not size, predicted which stream encodes a feature.
From the original article
1 min read at arxiv.orgAudio language models pass speech and text through a single decoder. We ask whether that decoder represents a distinctive feature in the same direction when a phoneme is heard and when it is read. For minimal pairs of phonemes differing in one feature, we take the offset between the two members' mean representations.
By Yuanhao Chen · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.