CertMoE certifies MoE LLM robustness under dynamic routing
Original title: CertMoE: Robustness Certification for Mixture-of-Experts LLMs via Routing-Aware Smoothing
Existing fixed-graph certificates were violated more than 30 percent of the time; routing-aware checks cut that to zero in tests.
CertMoE is a routing-aware robustness certification framework for mixture-of-experts large language models. Standard certificates assume a fixed computation graph, which input-dependent expert routing violates. The method checks that perturbed Top-K routing stays inside a larger certified expert pool, then certifies the induced sub-network. Experiments found existing certificates often violated, while CertMoE reduced the empirical violation rate from more than 30 percent to 0.
- Dynamic routing changes the certified object into input-dependent sub-functions.
- A relaxed routing-containment criterion bounds the perturbed expert pool.
- The result is a joint guarantee over routing and prediction consistency.
From the original article
1 min read at arxiv.orgProvable robustness is a cornerstone of trustworthy machine learning, aiming to guarantee model predictions under adversarial perturbations. Existing certification frameworks, inspired by abstract interpretation or symbolic execution, typically assume a fixed computation graph, where input perturbations or variations affect only model outputs but not the underlying execution path.
By Qingyue Wang · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.