Recognition-Refusal Misalignment in LLMs: Why Models Answer Structurally Unanswerable Questions
A linear hidden-state direction encodes question impossibility in 1.7B-70B LLMs, but misalignment with the safety-refusal pathway explains why models answer unanswerable questions.
The study examines why instruction-tuned LLMs from 1.7B to 70B parameters answer structurally unanswerable math and code questions instead of abstaining. A single linear direction in the hidden state separates answerable from impossible prompts, showing models represent impossibility before generation, but this direction is nearly orthogonal to the canonical safety-refusal direction. Generation-time steering along the recognition direction changes invalidity-aware behavior dose-responsively, and the geometry is present even at the pretraining endpoint, indicating a routing failure rather than an encoding failure.
- Models linearly represent question impossibility before generating answers.
- Recognition direction nearly orthogonal to canonical safety-refusal direction.
- Steering along recognition direction changes abstention behavior bidirectionally.
- Failure is a routing problem, not a missing internal signal.
- Low-cosine geometry already present at the pretraining endpoint.
Full article176 words · extracted from huggingface.co · click to collapse
Large language models often answer structurally unanswerable questions, such as computing cot(-540°) or evaluating (1).startswith("1"), instead of abstaining. We ask whether this failure reflects missing recognition or failed routing from recognition to abstention. Across instruction-tuned models from 1.7B to 70B parameters, a single linear direction in the hidden state separates answerable from structurally impossible math and code prompts, showing that models represent impossibility before generation. Yet this recognition direction is nearly orthogonal to the canonical safety-refusal direction that mediates trained harmful-content refusal. An in-domain behavior-defined invalidity-aware direction is closer to recognition, but only partially aligned with it, and remains near-orthogonal to safety refusal. Generation-time steering along the recognition direction changes invalidity-aware behavior bidirectionally and dose-responsively on structural math and code cells, while random directions do not. Base/instruct comparisons further show that the low-cosine geometry is already present at the pretraining endpoint. The confident-on-impossible failure is therefore better explained as a routing failure than as an encoding failure: the model has a usable "no admissible answer" signal, but the safety-refusal pathway is not aligned to use it.
Text extracted automatically; images, tables and formatting may be missing. Original: https://huggingface.co/papers/2608.29109