SEAM expands TTS vocabularies without paired audio fine-tuning
Original title: SEAM: Spectral Embedding Alignment and Merging for Low-Resource TTS
The method matches or approaches baselines with up to four times less data across three languages and two architectures.
SEAM is a zero-audio initialization method for expanding conditional text-to-speech vocabularies. It builds embeddings for unseen tokens from known subtokens, applies a spectral filter to keep phonetic order, and aligns them with ridge regression. Across three languages and two TTS architectures, it approaches or matches baselines while using up to four times less training data. The authors report a 75% reduction in sample complexity versus random initialization and mean pooling.
- Standard pooling is said to ignore sequential phonetic structure.
- Closed-form ridge regression aligns new embeddings with the pretrained manifold.
- Sample complexity falls 75% versus random initialization and mean pooling.
- The paper frames SEAM as lightweight and reproducible for low-resource speech.
From the original article
1 min read at huggingface.co
Expanding the vocabulary of conditional text-to-speech (TTS) models typically requires extensive fine-tuning with paired audio, creating a major bottleneck for low-resource and cross-lingual adaptation. To address this, we introduce SEAM, a zero-audio initialization method that constructs embeddings for unseen tokens from known subtokens.
By Walaa Othman, Baher Mohammad, Stamatios Lefkimmiatis · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.