Eigenvalues of the Hessian in Deep Learning: The Origin of Symmetry and Its Breaking
A symmetry-breaking account explains clustered Hessian spectra in trained deep networks, including ReLU, CNN, graph, and transformer models.
Stories per day · 30 days
2 stories, newest first · one card per story
Stories tagged #ntk
A Spectral Theory of Grokking: Weight Decay induces Feature Learning
Paper derives a spectral theory showing weight decay drives NTK feature learning that controls grokking timescale and phase structure.