Sheaf method isolates paraphrase-stable content in transformers
Original title: Gluing Local Contexts into Global Meaning: A Sheaf-Theoretic Decomposition of Transformer Representations
A 20-dimensional content subspace beats LEACE by a mean 17.1 points on held-out CounterFact retrieval across eight architectures.
The paper builds a cellular sheaf over a paraphrase graph so a frozen model's Laplacian can test whether activations keep the same content when phrasing changes. The kernel is treated as a content subspace, and its spectral complement is linked to causal influence and steering fragility. The same graph supplies the content subspace, influence directions, and a fragility ranking.
- Jointly constraining all paraphrases of a fact adds a further 6.5 points.
- Ablating paraphrase-variable coordinates changes outputs 5.6 to 26.5 times more.
- Tests cover nine models from 124M to 13B parameters.
- Harmonic mass ranks six architectures with steering fragility at Spearman 0.90.
From the original article
1 min read at huggingface.co
Existing interpretability tools do not test whether a representation keeps the same content when only the phrasing changes. We treat each paraphrase pair as a local witness that two activations should agree and build a cellular sheaf over the paraphrase graph; its Laplacian L_F answers the question on a frozen model with no training.
By Bryce Grant, Peng Wang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.