Paper finds reasoning models inherit 'wait' rather than learn it from RL
Original title: Inherited, Not Invented: Tracing the Provenance of Wait in Reasoning Models
On bases that lack the word, RL still raises MATH-500 accuracy while 'wait' stays nearly absent.
A paper traces the reasoning marker "wait" through open models' training histories using label-free frequency, position, form and usage indices plus an injected-error test. It finds zero-RL almost never coins the marker, while mid-training in OLMo-3 and fine-tuning on reasoning traces install it. Released RL-Zero recipes for OLMo-3 and MiMo multiply an existing marker, but KL-free GRPO from the same base does not, and the authors' GRPO recipe erases a marker seeded by fine-tuning.
Unconfirmed: Results are the authors' measurements on sampled outputs, with no independent replication described.
- RL after supervised fine-tuning barely changes an existing marker.
- OLMo-3's recipe makes the marker a more sensitive but more liberal alarm.
- Authors say marker frequency alone does not show RL-induced reasoning.
- They warn automated judges can reverse human-labelled revision trends.
From the original article
1 min read at huggingface.co
Reinforcement learning (RL) is credited with teaching reasoning models to write {wait} before reconsidering a step, yet large RL accuracy gains do not require the word. This view can confuse what RL discovers with what a model inherits from its training data.
By Zhizhuo Kou, Sizai Hou, Long Chen · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.