Multi-Dimensional Prosody Judgment For Live Streaming Speech Synthesis
Researchers introduce Live-ProsodyJudge and D-LPJ, Gemini-distilled Qwen3-Omni judges that decouple multi-dimensional prosody scores for live streaming TTS evaluation.
Researchers introduce Live-ProsodyJudge (LPJ), a pairwise TTS prosody evaluator distilled from Gemini into Qwen3-Omni for cost-effective live streaming speech synthesis evaluation. They identify verdict coupling, where multi-dimensional judges collapse dimension scores into a single preference bit, and propose Decoupled-Live-ProsodyJudge (D-LPJ) using masked SFT and a span-local GRPO strategy. Balanced-order LPJ beats a single Gemini call in point accuracy, and in a Best-of-8 TTS selection tournament the chosen utterance falls in the human top-3 for 85.29% of high-confidence cases.
- LPJ distills Gemini's pairwise prosody judgment into Qwen3-Omni for affordable large-scale TTS evaluation and RL feedback
- Identifies verdict coupling: multi-dimensional judges lazily align dimension scores with overall preference
- D-LPJ removes overall verdict targets, masks uncertain pair-dimensions, and applies span-local GRPO advantages
- In Best-of-8 selection, LPJ picks human top-3 utterances in 85.29% of high-confidence cases
- Organizations
- IBM
- AI models
- GeminiQwen3-Omni
From the original article
1 min read at arxiv.orgEvaluating live streaming speech synthesis (TTS) requires assessing fine-grained, highly expressive prosody such as emotion, intonation, and energy which traditional MOS predictors fail to capture. While proprietary Large Language Models (LLMs) like Gemini can evaluate these aspects, they are too costly for massive inference and reinforcement learning feedback.
By Zifan Guan · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.