Any ReLU network can be exactly simulated with two hidden layers
Original title: Two $\operatorname{ReLU}$ Layers is All You Need
The authors give a homological-algebra proof using fillings and a separate direct combinatorial proof.
A paper proves that every ReLU network can be exactly simulated by a ReLU network with two hidden layers. One argument uses fillings from homological algebra, and the other is a direct combinatorial proof. The abstract does not describe experiments, implementations, or applications.
- The claim is exact simulation, not approximation.
- Both proofs concern networks that use the ReLU activation.
From the original article
1 min read at arxiv.orgWe prove that every $\operatorname{ReLU}$ network can be exactly simulated by a $\operatorname{ReLU}$ network with two hidden layers. We present two proofs: one that is based on fillings in homological algebra and a direct combinatorial proof.
By Florestan Brunck · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.