CausalCoT isolates sub-questions to limit chain-of-thought attention leakage
Original title: Causal Chain of Thought: Mitigating Attention Leakage via Context Isolation
On reasoning benchmarks, causal sensitivity of intermediate steps rose 12 points on average and up to 21.
CausalCoT is an inference framework that turns a problem into a directed acyclic graph of atomic sub-questions and evaluates each node using only its direct causal parents. The design blocks attention to non-parent context so later answers depend on local reasoning rather than full-context lookup. The authors report competitive predictive performance, higher explanation faithfulness, and test-time corrections that propagate through the dependency graph.
- Each node is evaluated in an isolated context limited to its causal parents.
- Causal sensitivity of intermediate steps rose by an average of 12 percentage points.
- The largest gain versus the strongest baseline was 21 percentage points.
- Local corrections can be applied at test time and flow to downstream steps.
From the original article
1 min read at arxiv.orgChain-of-thought (CoT) prompting elicits step-by-step rationales that appear to explain model predictions, but there is no guarantee these steps causally drive the final answer. In standard autoregressive generation, dense attention allows each reasoning step to attend to the full context, enabling the model to derive answers through direct lookup while generating plausible-sounding but post-hoc justifications.
By Gabriele Dominici · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.