arXiv cs.AI / cs.LG / cs.CL·4d agoA Spectral Theory of Grokking: Weight Decay induces Feature Learning#grokking#feature-learning#ntkAI research
arXiv cs.AI / cs.LG / cs.CL·18d agoWhen Does Scale-Invariant Optimization Become Unstable? An Exact Schedule Law with Weight Decay#deep-learning-theory#learning-rate-schedules#normalizationAI research
arXiv cs.AI / cs.LG / cs.CL·19d agoA Theoretical Analysis of Generalization Dynamics in Neural Networks under Gradient Descent with Weight Decay#generalization#gradient-descent#grokkingAI research