Two-ReLU gradient descent collapses to bisector then specializes
Gradient flow on two-neuron ReLU model first collapses to bisector saddle before escaping to specialize to teacher.
Stories per day · 30 days
2 RSS
2 stories, newest first · one card per story
Stories tagged #neural-network-theory
Bias-Induced Crossover in Absolute Capacity of Dense Associative Memory
Analysis shows biased patterns cut dense associative memory capacity from N^(n-1)/ln N to O(N^(n/2)), with a bias-induced crossover.