Self-Preservation in Language Models
Qwen models can generate replay and sometimes choose tasks that better preserve prior capabilities.
Researchers study whether language models can choose preservation targets that protect capabilities more broadly during continual learning. They organize task descriptions and examples into Task Banks that a frozen model uses to generate replay without original training data. Continual instruction tuning of Qwen models of several sizes shows generated replay reduces forgetting on tasks outside the bank, and combining partial banks improves mean retention of previously correct answers. Selection informed by the new task improves mean held-out retention over random selection at 4B, though smaller models show no corresponding advantage.
- Task Banks store descriptions and examples that guide replay generation.
- Generated replay reduces forgetting on tasks outside the bank.
- Task-informed selection improves held-out retention for a 4B Qwen model.
- Replay under a fixed budget preserves more but slows new-task gains.
From the original article
1 min read at huggingface.co
Continual learning (CL) seeks to adapt pretrained models while preserving prior capabilities. Yet the tasks supplied for preservation cover only part of what a model knows, while forgetting can extend beyond them. Recent work shows that language models can supplement this limited evidence by generating useful replay without access to their original training data.
By Dongkyu Cho, Rumi Chunara, Ali Behrouz · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.