LACE: Layer-Wise Compression for Dynamic Frame Rate Codecs
LACE introduces layer-wise compression for dynamic frame rate audio codecs, cutting sequence lengths and speeding TTS inference while preserving quality.
LACE (Layer-Adaptive Codec Encoding) applies an independent compression step at each quantization layer of a neural audio codec, enabling layer-specific segmentation boundaries instead of shared ones. Union alignment and boundary anchor mechanisms keep durations consistent for downstream text-to-speech. On LibriTTS, LACE achieves a better rate-quality tradeoff than prior dynamic frame rate codecs and improves TTS inference efficiency at competitive synthesis quality. Code is released in the ESPnet3 codec recipe.
- Layer-specific compression replaces shared segmentation across quantization layers
- Union alignment and boundary anchors keep TTS durations consistent across layers
- Better rate-quality tradeoff than prior dynamic frame rate codecs on LibriTTS
- Code released as part of the ESPnet3 codec recipe
From the original article
1 min read at arxiv.orgNeural audio codecs are a key component in speech language modeling. However, their high frame rates lead to long sequence lengths, increasing computational costs. Dynamic frame rate codecs mitigate this by reducing the effective frame rate using a compression step to merge multiple frames together.
By Thanapat Trachu · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.