Muon optimizer doubles first MoE layer bias span
Muon raises router bias span in early MoE layers, complicating load balancing at tuned learning rates.
Stories per day · 30 days
1 stories, newest first · one card per story
Stories tagged #moe-balancing