From Redundancy to Minimality: Fixed-Point-Guided Hierarchical Reduction of Learned Piecewise-Linear Dynamics
Fixed-point-guided reduction turns redundant AL-RNNs into minimal dynamics, lifting Chua success from 20% to 71%.
Almost-linear RNNs use a limited set of ReLU units so nonlinear capacity is explicit, and activation patterns form a symbolic transition graph. Direct training with few ReLU units often fails to find minimal dynamics, so the authors train a larger model and then apply fixed-point-guided hierarchical reduction. Selected ReLU units are linearized, neighboring regions are merged, and each candidate is retrained from its parent while distinct fixed-point symbols are preserved. They prove that Q distinct fixed points require at least Q such symbols; on the 3-scroll Chua system, learn-reduce-retrain raises high-fidelity minimal success from 20% of seeds to about 71%.
- AL-RNNs limit nonlinearity to a chosen subset of ReLU units.
- Hierarchical reduction linearizes ReLU units while keeping fixed-point symbols.
- Reproducing Q distinct fixed points requires at least Q FP-containing symbols.
- On 3-scroll Chua, success rises from 20% of seeds to about 71%.
From the original article
1 min read at arxiv.orgUnderstanding a nonlinear dynamical system from time series requires not only reproducing its trajectories, but also identifying a simple representation that preserves its essential dynamical structure. Almost-linear recurrent neural networks (AL-RNNs) are piecewise-linear RNNs in which only a subset of units use ReLU nonlinearities, so that nonlinear capacity is explicitly controlled by the number of ReLU units.
By Hiroto Tamura · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.