Constructive Bounds for the Topological Expressivity of Neural Networks
A paper proves ReLU architectures can realize prescribed Betti numbers, with tighter bounds for products of spheres.
The authors study topological expressivity through Betti numbers of a network's output rather than a decision boundary. They prove a ReLU network whose widths depend on the target Betti numbers can represent spaces homotopy-equivalent to wedge sums of spheres. Constructions based on products of spheres give a tighter bound, and they conjecture that a sphere needs only a (d, d+1, d+1) architecture.
- Measures expressivity by Betti numbers of the network output.
- Proves ReLU widths sufficient for wedge sums of spheres.
- Products of spheres admit a tighter three-layer bound.
- Conjectures width d+1 suffices to represent a sphere.
From the original article
1 min read at arxiv.orgThe expressivity of a neural network $Φ$ describes the class of functions that a network can express. The term topological expressivity refers to a family of measures particularly suited to understand binary classification and is often formulated in terms of the Betti numbers of a decision boundary $β(Φ^{-1}(0))$.
By Kosio Beshkov · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.