Tutorial on Generative Models for Music Audio
Original title: A Tutorial on Generative Models for Music Audio
This monograph covers neural audio codecs, autoregressive and diffusion models, sampling, and preference optimization for music generation systems.
The paper explains methods for generating music audio including instrumental clips, songs with vocals, edits, and real-time performance. It introduces neural audio codecs, autoregressive and masked token models, diffusion and flow matching, sampling acceleration, and preference optimization. Sections connect these to musical requirements for instrumental, song, editing, and live performance generation. The text also covers interpreting metrics, listening tests, data provenance, and legal implications for training.
- Covers instrumental clips, full songs, edits, and live performance systems
- Introduces neural audio codecs and autoencoders
- Compares representations, methods, inputs, and tasks across papers
- Discusses metrics, listening tests, memorization, and attribution
- Includes symbolic composition, speech, and sound effects
- Organizations
- Hugging Face
From the original article
1 min read at huggingface.co
Music audio generation spans instrumental clips, full songs with vocals, edits of existing recordings, and systems that play alongside a musician in real time. This monograph explains the methods behind these systems for readers familiar with neural-network training who are new to music generation.
By Mateusz Modrzejewski, Joanna Kurkowska, Dorien Herremans · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.