DuplexCadence: Exact State and Execution from a Speech Model's Declared Timelines
DuplexCadence speeds full-duplex speech serving 2.85× and cuts peak memory 38.8% by exposing native clocks.
Full-duplex speech models must turn each second of input into speech before the next second arrives, but the token-to-audio synthesis tail overruns that cadence. DuplexCadence exposes the model's native clocks so the runtime can allocate demand-sized state at stable addresses and replay exact-shape graphs without padding. Across four released models and three decoder architectures, with bit-for-bit identical output, it reaches 2.85× the stock runtime's speed at 38.8% lower peak memory. Mean SPEAK time falls from 14% over the one-second cadence to 2% under it.
- Full-duplex speech serving must finish each second before the next arrives.
- Autoregressive stages fit the cadence; token-to-audio synthesis causes overruns.
- Native clocks enable stable demand-sized allocation and exact-shape graph replay.
- 2.85× stock speed and 38.8% lower peak memory with bit-identical output.
- Mean SPEAK time moves from 14% over the one-second cadence to 2% under it.
From the original article
1 min read at arxiv.orgFull-duplex speech models support streaming interaction that listens and speaks at the same time. Serving them is governed by a strict, repeating deadline: conversation advances on a one-second cadence, and every second of input must be turned into a second of speech before the next second arrives. Because stages within a session run in strict sequence, per-invocation overhead cannot be batched away.
By Haixiao Gao · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.