Thinking Inertia: LLMs Keep Thinking When Told Not To
Study finds LLMs keep producing explicit reasoning even when instructed not to think.
The paper studies whether LLMs can suppress visible reasoning when told not to think, arguing that disabled thinking modes and trace length are unreliable proxies. It scores responses with Empty-Thinking Rate, question-to-trace relevance, and an LLM-judge Explicit Inference Rate. Across six prompting interventions and six LLMs, explicit inference persists and increases as the answer space opens. Accuracy stays stable on Boolean and multiple-choice items, while open-ended tasks show a compliance-versus-accuracy trade-off.
- Three metrics separate answer-only output, relevant filler, and explicit inference.
- Six prompting interventions were tested on six LLMs.
- No-think controls do not reliably stop visible reasoning.
- Open-ended tasks trade answer-only compliance against accuracy.
- Supplying candidate answers makes answer-only responses easier.
From the original article
1 min read at huggingface.co
Large Language Models (LLMs) increasingly ship with explicit "thinking modes", yet their counterpart, "no-thinking", has received far less attention. We study LLMs' no-thinking behavior along two axes. a.
By Dianqiao Lei, Kevin Qinghong Lin, Pan Lu · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.