Drift-Constrained Optimization: Only Direction Matters in Fine-Tuning Instruct Models
Drift-Constrained Optimization reformulates fine-tuning as update-direction selection, letting Qwen3 models improve target tasks within a behavioral drift budget.
The paper specifies a behavioral drift budget before optimization and shows that update direction is the remaining degree of freedom, reformulating fine-tuning as a direction-selection problem. In a stringent QA-only setting where instruct models must still generate multi-step reasoning at inference, a coarse layer-selective probe reverses the failure of QA-only fine-tuning. Across Qwen3-8B and Qwen3-14B, these directions substantially improve scientific reasoning and multilingual translation, matching or outperforming dedicated translation systems over 100+ languages and giving stronger initialization for reinforcement learning.
- Reformulates fine-tuning as direction selection under a pre-set drift budget
- Layer-selective probe reverses QA-only fine-tuning failure while preserving reasoning
- Substantially improves scientific reasoning and multilingual translation on Qwen3-8B and Qwen3-14B
- Matches or outperforms dedicated translation systems across over 100 languages
From the original article
1 min read at huggingface.co
Fine-tuning instruct models often improves target performance while inducing behavioral drift from the reference model, which can degrade existing capabilities. Rather than treating this drift as an uncontrolled consequence of optimization, we specify a behavioral drift budget before optimization and ask how to boost the target-task performance within it.
By Fei Yuan, Changjiang Gao, Yilei Tu · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.