Intrinsic-Extrinsic Coupling in Learning Dynamics
Researchers show replay can erase a classifier-head intervention's gains, formalizing intrinsic-extrinsic coupling in learning.
The paper formalizes intrinsic-extrinsic coupling, arguing that current observations do not fully determine a learner's response to further training. An executable classifier-head write preserves current logits while repairing specified historical margins. In a CLINC-derived class-incremental setting, replay changed that write's 32-update contribution from five correct predictions to zero. Nonzero interactions also appeared under output distillation, a RoBERTa backbone, and SGDW, where correct-count interactions were negative at 128 updates.
- Coupling is measured by continuation-conditioned value of a state intervention.
- Replay cut a 32-update write from five correct predictions to zero on CLINC.
- Under SGDW, correct-count interactions were negative at 128 updates.
- Guided allocation beat standard replay on cross-entropy in all five pairs.
From the original article
1 min read at arxiv.orgA learner's current observations need not determine its response to further training. We formulate intrinsic-extrinsic coupling through the continuation-conditioned value of a constrained learning-state intervention, with observation-relative fibers describing present agreement.
By Qinyou Wang · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.