EmoRES-TTS: Residual-Enhanced Vector Steering for Emotional Speech Generation
EmoRES steers frozen TTS models and beats CoCoEmo on IEMOCAP emotion metrics.
EmoRES is a training-free vector-steering method for emotion-conditioned text-to-speech that separates an emotion vector into a shared component, which moves speech away from neutral, and a residual that targets the requested emotion. On IEMOCAP it beats CoCoEmo on all four objective emotion metrics for frozen IndexTTS-2 and CosyVoice2 backbones. Rank correlation improves by 26.13 and 12.97 percentage points, and emotion hit rate by 12.95 and 6.92 points. Human evaluation reports up to 35.0% relative gain in identifying the requested emotion, up to 17.3% better fidelity, and naturalness preference in up to 63.8% of pairwise comparisons.
- EmoRES splits emotion vectors into shared and residual components.
- Steering is training-free on frozen IndexTTS-2 and CosyVoice2.
- IEMOCAP rank correlation rises 26.13 and 12.97 points versus CoCoEmo.
- Emotion hit rate improves 12.95 and 6.92 points on the two backbones.
- Listeners preferred EmoRES for naturalness in up to 63.8% of pairs.
From the original article
1 min read at arxiv.orgEmotion-conditioned text-to-speech (TTS) models may fail to express the requested emotion reliably, and improving controllability by additional training is costly in both computation and emotion-labeled speech training data. We therefore study vector steering, a training-free approach that modifies the internal representations of a frozen model.
By Kuan-Po Huang · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.