Train Smarter, Not Harder: Switching Signal-Guided Training in Active Learning
HybridAL is an active-learning training schedule that switches from retraining to fine-tuning on stabilization signals, saving up to 49% time.
Researchers find that choosing between retraining from scratch and fine-tuning is an exploitable decision variable in active learning: retraining helps in early rounds while fine-tuning is safer once the model trajectory stabilizes. HybridAL monitors an online stabilization signal using spectral exponent change and accuracy change, switching from retraining to fine-tuning after sustained stabilization. Across three encoder backbones and six text-classification tasks with five seeds each, HybridAL keeps endpoint macro-F1 non-inferior within a 0.010 margin, saves up to 49% of retraining time, and improves the time-calibration trade-off measured by negative log-likelihood.
- Shows retraining versus fine-tuning choice is exploitable in active learning
- HybridAL switches from retraining to fine-tuning after sustained stabilization
- Uses spectral exponent change and accuracy change as switching signals
- Saves up to 49% of retraining time with non-inferior macro-F1
- Outperforms fixed-round switching on NLL at moderate extra cost
- Products
- HybridAL
From the original article
1 min read at huggingface.co
Training strategy, namely whether to retrain from scratch or fine-tune from the previous checkpoint, is an overlooked decision variable in active learning. We show that this choice has exploitable structure: retraining is most useful in early rounds, when each batch can substantially reshape the labeled distribution, while fine-tuning becomes safer once the model trajectory stabilizes.
By Nagham Omar, Maya Rozenshtein, Evgeny Mishlyakov · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.