How Does Distribution Shift Shape Pretraining Gains in Neural PDE Surrogates?
Study shows pretraining gains for neural PDE surrogates depend on distribution shift type, target data budget, and whether source-target physics differ.
The authors pretrain a neural PDE surrogate on 254,909 RANS solutions from one airfoil family and fine-tune it on a new family under two target settings: same Spalart-Allmaras modeling and SA with added e^N transition modeling. At a 1000-sample budget, the pretrained model matches a from-scratch model trained on 3.25x as many samples for the same-physics target, but only 2.58x for the transition-modeled target, with orderings reversing at 5000 samples. Results indicate pretraining value depends jointly on target budget, coverage, and differences in modeled physics between source and target.
- Pretraining on 254,909 RANS airfoil solutions reduced data needs on a same-physics target
- Pretraining gains shrink when target physics changes (added e^N transition modeling)
- Benefit ordering between targets reverses as the fine-tuning budget grows to 5000 samples
- Sampling more distinct airfoils lowered error, but gains exceeded variance only for same-physics transfer
From the original article
1 min read at arxiv.orgPretraining a neural PDE surrogate can reduce the amount of new CFD data needed when geometry or modeled physics changes. However, it remains unclear how different components of distribution shift affect this benefit.
By Pochinapeddi Sai Bhargav · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.