Mesoscopic theory describes SGD feature learning in wide two-layer networks
Gaussian overlap statistics track linear and quadratic learning until specialization, which quadratic shielding accelerates.
Stories per day · 30 days
2 stories, newest first · one card per story
Stories tagged #stochastic-gradient-descent
Quantile-based Loss Filtering for Outlier-Robust Stochastic Gradient Descent
Quantile-k-Loss SGD filters corrupted component losses by quantile sampling, proving linear convergence while outperforming standard and min-k-loss SGD.