UniAE-MoE: A Unified Audio Encoder via Mixture of Experts
UniAE-MoE fuses Qwen2-Audio and Audio-Flamingo 3 encoders, scoring 0.802 on XARES-LLM.
UniAE-MoE is a mixture-of-experts audio encoder that fuses strong encoders from Qwen2-Audio and Audio-Flamingo 3 for speech, music, and general audio. The design uses SwiGLU with shared experts, a two-stage instruction-tuning strategy, and task-specific data scaling. It scores 0.802 on XARES-LLM, reported as state of the art, and ranks at the top of the official Interspeech 2026 Audio Encoder Capability Challenge.
- Fuses encoders from Qwen2-Audio and Audio-Flamingo 3
- Uses SwiGLU shared experts and two-stage instruction tuning
- Task-specific data scaling improves downstream understanding
- Scores 0.802 on XARES-LLM, claimed state of the art
- Top-tier in the Interspeech 2026 audio encoder challenge
- AI models
- UniAE-MoEQwen2-AudioAudio-Flamingo 3
From the original article
1 min read at arxiv.orgLarge Audio Language Models (LALMs) rely on effective audio encoders for multi-task performance. We introduce UniAE-MoE, a unified audio encoder designed to model cross-domain audio representations and achieve outstanding downstream understanding performance via a Mixture-of-Experts (MoE) architecture.
By Shengbo Cai · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.