DEFINE: Exemplar-Guided Accent Control for Zero-Shot TTS
DEFINE decouples speaker identity from accent in zero-shot TTS using separate audio exemplars, with a single inference-time guidance weight controlling accent strength.
DEFINE is an end-to-end framework built on F5-TTS with LoRA adaptation that conditions speaker identity and target accent on separate audio exemplars, decoupling the two factors usually entangled in one reference. An exemplar encoder supervised via learned accent prototypes maps short accent exemplars into a conditioning space, requiring no accent labels or post-synthesis waveform conversion at inference. On seen accents, raising accent guidance improves accent-probe accuracy from 6.5% to 19.6%, and a single model matches a two-model TTS-voice-conversion cascade on seen and out-of-domain accents while achieving higher speaker similarity.
- Separate exemplars control speaker identity and accent; guidance weight tunes accent strength without retraining
- Built on F5-TTS with LoRA; accent encoder uses learned prototypes, no inference-time labels
- Accent-probe accuracy rises 6.5% to 19.6% on seen accents; matches TTS-VC cascade with higher speaker similarity
From the original article
1 min read at huggingface.co
Zero-shot text-to-speech (TTS) can reproduce an unseen speaker from a short reference recording, but typically entangles speaker identity and accent within the same reference. We introduce DEFINE, an end-to-end framework that decouples these factors by conditioning speaker identity and target accent on separate audio exemplars.
By Ambuj Mehrish, Abhinaba Roy, Alex Ivanov · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.