Mothersuperior/yue2-mothersuperior-realaudio-tokenizer-v4 — new model trending #30 on Hugging Face
Community release ships a real-audio semantic tokenizer and NAR LoRA for YuE2-3B, enabling artist-style song generation and covers from user recordings.
A Hugging Face community release (trending #30) provides an audio-to-semantic-token encoder for YuE2-3B: an 8-layer transformer (d=512) mapping MERT-v2-FullSong layer-20 features at 25 Hz to 32,768 YuE2 semantic codes, with 16.1% held-out top-1 exact match. It also includes a rank-32 LoRA over the NAR branch's 28 layers plus full vae2llm/llm2vae replacement weights, trained on 4,765 YuE2-generated songs then adapted to real audio. Weights ship in .pt, safetensors, bf16, and a ComfyUI-native LoRA layout, with training/inference scripts and an optional YuE2-Vae decoder weight merge (0.666/0.334). Full pipeline runs on a 24 GB GPU (14–18 GB measured with gradient checkpointing).
- Tokenizer head maps MERT-v2-FullSong layer-20 features to 32,768 YuE2 semantic codes at 25 Hz.
- Held-out exact-match top-1 is 16.1%; NAR round-trip ear tests sit around 95%.
- Rank-32 NAR LoRA spans 28 layers; vae2llm/llm2vae ship as full replacement weights.
- Includes ComfyUI-compatible LoRA and optional YuE2-Vae decoder merge (0.666 default / 0.334 legacy).
- AR LoRA training uses a 4,732-song regularizer pack; over ~1,500 steps the model memorizes songs.
- Organizations
- m-a-pMothersuperior
- AI models
- YuE2-3B
From the original article
7 min read at huggingface.coReal-audio tooling for [YuE2-3B](https://huggingface. co/m-a-p/YuE2-3B): the **audio → semantic-token encoder** YuE2 doesn't ship, plus a **NAR-branch LoRA** so the decoder renders real-production latents. Together they let you tokenize your own recordings, LoRA-tune YuE2's AR on | `tokenizer_head_joint_v4.
By Mothersuperior · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.