The Thinking Leakage in Reinforcement Learning for LLMs
Qwen3 NO-THINK RL gains largely reactivate existing THINK-mode reasoning rather than new skills.
The authors report that GRPO training of Qwen3-1.7B, 4B, and 8B in NO-THINK mode improves reasoning largely by reactivating the original THINK mode, which they call thinking leakage. As accuracy rises, responses become longer and more reflective, and internal activations shift toward THINK mode. Further scaling approaches but does not exceed THINK-mode accuracy. In Qwen3-8B, replacing a low-dimensional THINK-related activation subspace removes most of the gain, while the reverse transfer moves much of it to the original model.
- Qwen3-1.7B, 4B, and 8B were trained with GRPO in NO-THINK mode.
- Accuracy gains coincide with longer, more THINK-like responses and activations.
- A subspace swap in Qwen3-8B removes most of the RL gain.
- Scaling the update approaches but does not exceed original THINK-mode accuracy.
- Vendors
- Alibaba
- Organizations
- Alibaba
- AI models
- Qwen3-1.7BQwen3-4BQwen3-8B
From the original article
1 min read at huggingface.co
Hybrid reasoning models such as the Qwen3 series switch between a THINK mode and a NO-THINK mode through a few chat-template tokens. A popular recipe trains such models with reinforcement learning in NO-THINK mode and reports improved reasoning performance.
By Wenxuan Zhu, Shuming Liu, Bang An · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.