LACE: Layer-Wise Compression for Dynamic Frame Rate Codecs
LACE introduces layer-wise compression for dynamic frame rate audio codecs, cutting sequence lengths and speeding TTS inference while preserving quality.
LACE (Layer-Adaptive Codec Encoding) applies an independent compression step at each quantization layer of a neural audio codec, enabling layer-specific segmentation boundaries instead of shared ones. Union alignment and boundary anchor mechanisms keep durations consistent for downstream text-to-speech. On LibriTTS, LACE achieves a better rate-quality tradeoff than prior dynamic frame rate codecs and improves TTS inference efficiency at competitive synthesis quality. Code is released in the ESPnet3 codec recipe.
- Layer-specific compression replaces shared segmentation across quantization layers
- Union alignment and boundary anchors keep TTS durations consistent across layers
- Better rate-quality tradeoff than prior dynamic frame rate codecs on LibriTTS
- Code released as part of the ESPnet3 codec recipe
Full article183 words · extracted from arxiv.org · click to collapse
Neural audio codecs are a key component in speech language modeling. However, their high frame rates lead to long sequence lengths, increasing computational costs. Dynamic frame rate codecs mitigate this by reducing the effective frame rate using a compression step to merge multiple frames together. However, most prior methods either operate on single-codebook codecs or apply a single compression step before multi-layer quantization. This forces all quantization layers to share the same segmentation boundaries, despite the residual embeddings at different quantization layers exhibiting different rates of change over time. We propose LACE (Layer-Adaptive Codec Encoding), a dynamic frame rate codec that applies an independent compression step at each quantization layer, enabling layer-specific segmentation boundaries. To use LACE tokens in downstream text-to-speech (TTS), we further introduce union alignment and boundary anchor mechanisms to make durations consistent across layers while preserving compression benefits. Experiments on LibriTTS show that LACE offers a better rate-quality tradeoff than prior dynamic frame rate methods on the reconstruction task and improves TTS inference efficiency while maintaining competitive synthesis quality. Our code is released as part of the ESPnet3 codec recipe.
Text extracted automatically; images, tables and formatting may be missing. Original: https://arxiv.org/abs/2609.17509