Hippocampal consolidation without replay in fully fine-tuned networks
Fully trainable networks maintain learning across tasks using hippocampal indexing without replay or distillation.
The method indexes input geometry per group of synapses after each task. This limits synapse movement while allowing the whole network to keep learning. It cuts absorption of old knowledge by near-duplicates several-fold and outperforms frozen-backbone baselines.
- Memory indexes input geometry including residual energy.
- Cuts near-duplicate absorption several-fold.
- Works across four architecture families up to 8B parameters.
- Leads frozen-backbone baseline by 17.5 points on TACRED.
From the original article
1 min read at arxiv.orgNetworks that keep learning overwrite what they know. Prevailing remedies replay stored data, distil from old models or freeze the backbone. The brain suggests a different division of labour: the hippocampus indexes where experience lies in cortex while the cortex keeps learning.
By Keuntae Kim · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.