30
57
45
30
30
30
30
57
30
Nearly Tight Rademacher Bounds for Sparsely Activated Neural Networks
Theory paper derives nearly tight Rademacher complexity bounds for sparsely activated one-hidden-layer ReLU networks.
Building on Awasthi et al. (COLT 2024), the authors bound statistical complexity for networks where each input activates at most k of s hidden units. A support-preserving cover and normalized chaining argument remove the explicit dimension factor, with matching lower bounds up to logarithms. They also derive agnostic minimax excess-risk bounds of order min{1, sqrt(s/(km))} for a normalized bounded loss and show bias bounds comparable to WR restore worst-case rates even on domains where sparsity holds globally.
16
35
30
60
60
60
60
60
60
57
60
57
60
57
30
30
57
30
57
30
30
30
57
30
30
55
30
30
60
45
30