ECHO: A Matched-Contrast Benchmark for Context-Sensitive Turn-Taking in Full-Duplex Dialogue
ECHO benchmark pairs identical-overlap Chinese dialogue examples with contrasting contexts, revealing most full-duplex systems bias toward yielding the floor.
ECHO is a paired diagnostic benchmark for Chinese full-duplex turn-taking that matches examples with identical overlap transcripts but contrasting preceding multi-turn contexts, one requiring Yield and the other Keep. It also includes off-talk examples for diagnosing unnecessary yielding and introduces pair accuracy, which grants no credit to constant-action policies. Experiments across multiple full-duplex systems show most exhibit a pronounced Yield bias, performing substantially better on interruptions than backchannels. The benchmark and metadata will be publicly released.
- Paired examples share overlap transcript but require opposite Yield/Keep decisions
- Pair accuracy metric defeats constant-action policies
- Most systems biased toward Yield; better on interruptions than backchannels
- Off-talk examples diagnose unnecessary yielding
From the original article
1 min read at arxiv.orgFull-duplex spoken dialogue systems must distinguish interruptions that require yielding the floor from backchannels that permit continued speaking. Existing benchmarks typically evaluate events independently and may therefore reward fixed action preferences rather than context-sensitive decisions. We introduce ECHO, a paired diagnostic benchmark for Chinese full-duplex turn-taking.
By Shuofeng Zhao · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.