Caterpillar Net represents maxima with two hidden layers
Original title: Two hidden layer suffice: the caterpillar net
The nonrecursive construction uses hidden-layer widths O(N squared) and 2 to the O(N log N).
The paper studies maximum representation for continuous piecewise linear functions in ReLU networks, noting that a conjecture of logarithmic depth for maxima has been disproved. Inspired by a 2026 proof that two hidden layers suffice for general continuous piecewise linear functions, it introduces the Caterpillar Net. The construction represents the maximum of N inputs with two hidden layers of widths O(N^2) and 2^{O(N log N)} and has a direct geometric interpretation.
- The construction is nonrecursive.
- It targets the maximum of N inputs.
- Two hidden layers are used.
- The authors cite Brunck et al. (2026) as inspiration.
From the original article
1 min read at arxiv.orgEvery continuous piecewise linear (CPWL) function is a finite signed sum of maxima of affine functions, making maximum representation a central problem in the study of the expressiveness of ReLU networks. A now-disproved conjecture predicted logarithmic depth for maxima. Inspired by the proof by Brunck et al.
By Jakob Paul Zimmermann · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.