Latent-MoE: Domain-Aware Mixture-of-Experts for PDEs with Multi-Regime Physics
Latent-MoE adds domain-aware mixture-of-experts routing to PINNs, improving accuracy over an order of magnitude on multi-regime physics PDEs.
The paper shows standard coordinate networks used in physics-informed neural networks have translation-variant NTKs causing long-range gradient conflicts on PDEs with spatially varying physics. MoE architectures with centered compact-support routers produce a uniformly banded NTK that localizes learning. Latent-MoE interleaves domain-aware MoE blocks in a shared backbone, outperforming FB-PINNs and X-PINNs by over an order of magnitude on multi-stage time-variable physics benchmarks.
- NTK analysis explains PINN failure on multi-regime PDEs
- Compact-support routers yield exponentially decaying kernel-regression weights
- Latent-MoE allows capacity sharing across regions via shared backbone
- Improves over an order of magnitude versus global and rigid-decomposition baselines
- Products
- Latent-MoEFB-PINNsX-PINNs
From the original article
1 min read at arxiv.orgPhysics-informed neural networks (PINNs) struggle on PDEs whose governing physics varies across the domain. We trace this to a structural property of standard coordinate networks: their neural tangent kernel (NTK) is translation-variant and lets training points of large coordinate magnitude disproportionately influence predictions elsewhere, producing long-range coupling and gradient conflict during training.
By Hanwen Wang · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.