Knowledge That Is Linear Before Fine-Tuning Generalizes After It
How linearly readable a pretrained fact is strongly predicts whether fine-tuning on it generalizes.
Researchers distinguish active facts, which support generalization, from inert facts that can be recalled but do not transfer. They relabel existing relations, fine-tune on a subset of subjects, and test held-out subjects that share the original answer. Across eleven language models and 70 relations, linear readability of the original relation predicts this generalization at Pearson r of 0.87 to 0.92 within each model. The association holds after controls for data size, original accuracy, and subject frequency, and is robust to new labels and seeds.
- Active facts support generalization; inert facts can be recalled but do not transfer.
- Within-model correlation of linear readability and generalization is Pearson r 0.87 to 0.92.
- Result holds for eleven models and 70 relations after controls and relabeling.
- Recovered wrong labels match linear-probe answers more often than the pretrained model's.
From the original article
1 min read at arxiv.orgNot all facts a language model can recall support further learning equally well. We distinguish active facts, which support generalization beyond their own recall, from inert facts, which can be recalled but do not support such generalization in a given task. We study this distinction by constructing new relations from existing ones.
By Faridoun Mehri · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.