Recognition-Refusal Misalignment in LLMs: Why Models Answer Structurally Unanswerable Questions
A linear hidden-state direction encodes question impossibility in 1.7B-70B LLMs, but misalignment with the safety-refusal pathway explains why models answer unanswerable questions.
The study examines why instruction-tuned LLMs from 1.7B to 70B parameters answer structurally unanswerable math and code questions instead of abstaining. A single linear direction in the hidden state separates answerable from impossible prompts, showing models represent impossibility before generation, but this direction is nearly orthogonal to the canonical safety-refusal direction. Generation-time steering along the recognition direction changes invalidity-aware behavior dose-responsively, and the geometry is present even at the pretraining endpoint, indicating a routing failure rather than an encoding failure.
- Models linearly represent question impossibility before generating answers.
- Recognition direction nearly orthogonal to canonical safety-refusal direction.
- Steering along recognition direction changes abstention behavior bidirectionally.
- Failure is a routing problem, not a missing internal signal.
- Low-cosine geometry already present at the pretraining endpoint.
From the original article
1 min read at huggingface.co
Large language models often answer structurally unanswerable questions, such as computing cot(-540°) or evaluating (1). startswith("1"), instead of abstaining. We ask whether this failure reflects missing recognition or failed routing from recognition to abstention.
By Yucheng Du, Xiyang Hu · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.