Any ReLU network can be exactly simulated with two hidden layers
The authors give a homological-algebra proof using fillings and a separate direct combinatorial proof.
Stories per day · 30 days
Relatedneural-networks3relu2betti-numbers1deep-learning1homotopy1piecewise-linear1relu-networks1theory1topology1
3 stories, newest first · one card per story
Stories tagged #expressivity
Caterpillar Net represents maxima with two hidden layers
The nonrecursive construction uses hidden-layer widths O(N squared) and 2 to the O(N log N).
Constructive Bounds for the Topological Expressivity of Neural Networks
A paper proves ReLU architectures can realize prescribed Betti numbers, with tighter bounds for products of spheres.