Base Models Can Reason By Taking a Cue From Training Data
Fixed starting-token cues let base models recover much of the math and coding gains from RL training.
The paper finds that particular tokens at the start of a base model's reply can elicit reasoning competitive with reinforcement-learning-trained counterparts. One cue lifts Olmo-3-7B MATH-500 pass@1 from 42% to 78%, and another lifts Qwen3-14B from 72% to 87%. Causal edits of training data can turn an arbitrary word into a reasoning cue or remove an existing cue's effect, and can make a nonsense instruction work like step-by-step prompting. A safety case study shows different cues produce distinct refusal and compliance behavior corresponding to different training-document types.
56