StepAudio 3 Music Technical Report
StepAudio 3 Music introduces long-form text-controlled music generation using ABC-notation planning and flow-matching diffusion, ranking near the top music arena.
StepAudio 3 Music generates long-form, text-controlled music using a 50-Hz single-codebook tokenizer with 65,536 entries and a flow-matching diffusion Transformer over VAE latents. A Mixture-of-Experts autoregressive model first plans an arrangement in ABC notation (ABC-CoT) before predicting music tokens. With DPO fine-tuning, it tops AudioBox content and production quality scores and reaches Quality Elo 1105 on the Artificial Analysis Music Arena, behind Suno V5.5 and Mureka. Generation covers songs, accompaniment from dry vocals, and cover synthesis up to 5 minutes 30 seconds at 48-kHz output.
- Tokenizer encodes audio as a 50-Hz stream from a 65,536-entry single codebook.
- Mixture-of-Experts autoregressive planner emits ABC-notation arrangement plans before token generation.
- Flow-matching DiT predicts VAE latents decoded to 48-kHz audio for tracks up to 5:30.
- Reaches Quality Elo 1105 on the Artificial Analysis Music Arena, behind Suno V5.5 and Mureka.
From the original article
1 min read at huggingface.co
We introduce StepAudio 3 Music, a large-scale, long-form music generation model that supports explicit musical planning and open-domain text-controlled generation. The StepAudio Music Tokenizer represents audio as a 50-Hz stream from a 65536-entry single codebook, using semantically informed self-supervised and multi-task training to preserve musical structure and reconstruction-relevant information.
By Chengli Feng, Zhiyue Wu, Jiahao Song · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.