HeirCommit and EvoHeir filter updates in self-evolving language models
Original title: Selection Is Not Learning: Choosing Model Futures
HeirCommit cut harmful accepted updates 68-83% versus random inheritance; EvoHeir raised the math average 5.3% over Agent0.
Self-evolving loops such as R-Zero, Agent0 and Skill Self-Play inherit every update from tasks that look learnable. The authors say most updates on verified tasks do not make correct answers more likely. HeirCommit freezes an update and uses fresh questions to inherit or reject it. EvoHeir separates parameter inheritance from experience inheritance and led math and general-reasoning averages across four backbones.
Unconfirmed: The four backbone models are not named.
- A task that looks learnable need not produce a useful model update.
- Regret is decomposed into selection, absorption and confirmation errors.
- Experience can persist even when the model that produced it is not inherited.
- The 5.3% math gain is relative to the strongest baseline, Agent0.
From the original article
1 min read at huggingface.co
Self-evolving language models train on tasks they generate, so every round must decide which updated model the next round inherits. Current loops such as R-Zero, Agent0, and Skill Self-Play make this decision through the task, admitting tasks that look learnable and inheriting every resulting update.
By Haoyi Yang, Kshitij Mishra, Nils Lukas · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.