Fully polynomial algorithm learns positive linear combinations of ReLUs
Original title: Learning Positive Linear Combinations of ReLUs in Fully Polynomial Time
The method uses poly(d, k, 1/ε) samples and time under Gaussian inputs and L2 loss, without well-conditioned weight assumptions.
The paper gives the first fully polynomial-time PAC algorithm for learning positive linear combinations of k arbitrary ReLU activations on R^d under the Gaussian distribution in L2 loss. It returns a hypothesis ε-close to the target with high probability. Prior methods needed super-polynomial time in k or 1/ε, or strong conditioning on the weight matrix. The approach boosts a list-decodable weak learner from Hermite-tensor filtering, then refines weights on the sphere and prunes irrelevant vectors.
- Sample and runtime bounds are polynomial in d, k, and 1/ε.
- A weak learner recovers hidden weights to constant error, list-decodably.
- Implicit tensor methods build a low-degree filter from Hermite tensors.
- Gradient-style updates on the Euclidean sphere improve weight estimates.
- Pruning keeps only vectors that meet the target accuracy.
From the original article
1 min read at arxiv.orgWe study the fundamental problem of PAC learning positive linear combinations of $k$ arbitrary ReLU activations on $\mathbb{R}^d$ with respect to the Gaussian distribution, in $L_2$ loss. We give the first fully-polynomial time algorithm for this learning task.
By Ilias Diakonikolas · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.