Conversation Is a Two-Body Problem: Dyadic Evaluation of Full-Duplex Dialogue Models
DyaFDB scores both sides of 7,560 full-duplex model conversations instead of using a fixed examiner.
The paper argues that full-duplex spoken dialogue models are usually evaluated against pre-recorded audio or fixed examiners, capturing only one side of turn-taking. DyaFDB instead has two models converse under assigned roles with cooperative or conflicting goals, then scores both sides offline. Four tasks comprise 140 scenarios and 7,560 conversations across six self- and cross-play pairings. The experiments show that each model's behavior continually reshapes its partner, and the authors plan to release scenarios, role prompts, and recording protocols without pre-recorded audio.
- Single-sided tests miss joint turn-taking, overlap, and interruption.
- DyaFDB pairs two full-duplex models and scores both offline.
- Four tasks cover 140 scenarios and 7,560 recorded conversations.
- Each model's behavior continually reshapes how its partner responds.
From the original article
1 min read at arxiv.orgFull-duplex spoken dialogue models listen and speak at the same time, enabling voice agents to have natural, low-latency interactions that turn-based systems cannot offer. However, they are commonly evaluated against single-sided interlocutors: pre-recorded audio that cannot react, or an automated examiner that reacts in real time but only administers a fixed sequence of tests and is never graded.
By Sungnyun Kim · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.