Modest TFNO tuning beats published Well neural-operator baselines
Original title: Performance of Neural Operator Baselines on 'The Well' Dataset
With one input timestep and the same 12-hour limit, tuned TFNO lowers VRMSE on 15 of 17 tasks.
The Well is a 15-terabyte benchmark spanning 16 physical simulation domains for machine-learning surrogates. The authors say the released neural-operator baselines were cheap illustrative runs with little or no hyperparameter tuning and a 12-hour per-run wall-clock limit, yet later treated as fully tuned benchmarks. A short, non-exhaustive search for the Tucker-factorized Fourier Neural Operator, using only one input timestep, reduces VRMSE below both published FNO and TFNO baselines on 15 of 17 tasks, sometimes by several orders of magnitude. They conclude those released baselines should not be treated as architectural benchmarks.
Unconfirmed: The hyperparameter search is described as short and non-exhaustive, so the tuned results are not a full optimum.
- The Well covers 16 physical simulation domains and is 15 terabytes.
- Published baselines used a history of up to four input timesteps.
- This search uses a single input timestep under the same 12-hour limit.
- VRMSE falls by several orders of magnitude on some of the 17 tasks.
From the original article
1 min read at arxiv.orgThe Well, a 15-terabyte dataset spanning 16 physical simulation domains, provides an extensive benchmark for developing and evaluating machine learning surrogate models for complex physical systems.
By Yash Pincha · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.