Learning rate decay can exponentially speed SGD on some nonconvex functions
Authors prove a benchmark class where decay uses exponentially fewer gradient queries than any fixed small step size.
Stories per day · 30 days
1 stories, newest first · one card per story