Routing Drift Alone Does Not Diagnose Failure in Merged MoE LLMs
Researchers find MoE routing drift after merging does not reliably indicate routing failure.
46
Researchers find MoE routing drift after merging does not reliably indicate routing failure.
Routing drift after MoE merging does not reliably indicate failure; source-route restoration often lacks task benefit.
Today 1.6K readers · 2.5K views · 779 reads · all time 189K views