CoSTA compresses speech tokens for faster spoken dialogue models
Original title: CoSTA: Compressed Speech Token Adaptation for Efficient Spoken Dialogue Models
On bilingual benchmarks, generation speed rises about 1.7 times while response quality is preserved.
Spoken dialogue models often share one speech-token space for generation and synthesis, producing long autoregressive sequences. CoSTA is a post-hoc method that adds compressed generation tokens through vocabulary extension, pronunciation adaptation, and dialogue adaptation, then expands them back to the original tokenizer before synthesis. The original speech tokenizer and decoder stay in place. The authors report about 1.7 times faster speech-token generation, lower full-response latency, and higher throughput, with cross-architecture transfer.
- CoSTA keeps the original speech tokenizer and decoder.
- Compressed tokens expand deterministically before synthesis.
- Authors report lower latency and higher throughput with preserved quality.
- Further tests show complementarity with low-rate tokenization.
From the original article
1 min read at huggingface.co
Spoken Dialogue Models (SDMs) typically share a speech-token space for generation and synthesis, leading to long autoregressive sequences and costly inference. Lower-rate speech tokenizers shorten sequences but may introduce compression-quality trade-offs and costly replacement without breaking this coupling.
By Yingzhuo Deng, Chong Li, Bingxin Liu · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.