Specialists Learn Faster: Knowledge Organization in Pre-training Shapes Knowledge Update Efficiency
Models pretrained on chronological knowledge update faster than shuffled generalists, and a derived law predicts that cost.
Stories per day · 30 days
2 stories, newest first · one card per story
Stories tagged #data-ordering
Curriculum Learning as Transport: Understanding Curricula with Wasserstein Geodesics
Researchers model curriculum learning as Wasserstein transport over difficulty distributions, finding curriculum benefits are strongly task- and budget-dependent with no dominant strategy.