Multi-Dimensional Prosody Judgment For Live Streaming Speech Synthesis
Researchers introduce Live-ProsodyJudge and D-LPJ, Gemini-distilled Qwen3-Omni judges that decouple multi-dimensional prosody scores for live streaming TTS evaluation.
Researchers introduce Live-ProsodyJudge (LPJ), a pairwise TTS prosody evaluator distilled from Gemini into Qwen3-Omni for cost-effective live streaming speech synthesis evaluation. They identify verdict coupling, where multi-dimensional judges collapse dimension scores into a single preference bit, and propose Decoupled-Live-ProsodyJudge (D-LPJ) using masked SFT and a span-local GRPO strategy. Balanced-order LPJ beats a single Gemini call in point accuracy, and in a Best-of-8 TTS selection tournament the chosen utterance falls in the human top-3 for 85.29% of high-confidence cases.
- LPJ distills Gemini's pairwise prosody judgment into Qwen3-Omni for affordable large-scale TTS evaluation and RL feedback
- Identifies verdict coupling: multi-dimensional judges lazily align dimension scores with overall preference
- D-LPJ removes overall verdict targets, masks uncertain pair-dimensions, and applies span-local GRPO advantages
- In Best-of-8 selection, LPJ picks human top-3 utterances in 85.29% of high-confidence cases
Full article198 words · extracted from arxiv.org · click to collapse
Evaluating live streaming speech synthesis (TTS) requires assessing fine-grained, highly expressive prosody such as emotion, intonation, and energy which traditional MOS predictors fail to capture. While proprietary Large Language Models (LLMs) like Gemini can evaluate these aspects, they are too costly for massive inference and reinforcement learning feedback. To address this, we first introduce Live-ProsodyJudge (LPJ), a cost-effective pairwise evaluator distilled from Gemini into Qwen3-Omni. However, we identify a critical flaw in standard multi-dimensional evaluation: verdict coupling. The judge tends to lazily align all individual dimension scores with its overall preference, collapsing a rich multi-dimensional rubric into a single preference bit. To resolve this, we further propose Decoupled-Live-ProsodyJudge (D-LPJ). D-LPJ eliminates the overall verdict target to prevent blind following, masks uncertain pair-dimensions during Supervised Fine-Tuning(SFT), and introduces a novel span-local GRPO strategy that applies normalized advantages strictly to their corresponding rationale spans. Evaluated on highly curated human-annotated test sets, 10 sample balanced-order LPJ achieves higher point accuracy than a single Gemini call, while D-LPJ successfully produces independent,decoupled dimension judgments. Furthermore, in a Best-of-8 TTS candidate selection tournament, the LPJ-selected utterance falls within the human top-3 in 85.29% of high-confidence cases, demonstrating its efficacy for fine-grained TTS preference optimization.
Text extracted automatically; images, tables and formatting may be missing. Original: https://arxiv.org/abs/2609.20124