Causal robustness margin proposed to measure mechanistic LLM uncertainty
Original title: Measuring Mechanistic Uncertainty in Large Language Models via Causal Robustness
Incorrect or uncertain answers had smaller margins across three open-source models and five QA datasets.
The authors define mechanistic uncertainty as instability in the internal mechanisms supporting a response. They operationalize a causal robustness margin, the minimum internal intervention that changes the output, using Jacobian Lens concept steering vectors. They say the margin complements conventional output-based uncertainty methods. In a controlled knowledge-injection experiment, correctly learned facts showed larger margins after training.
Unconfirmed: Model names and dataset names are not stated in the excerpt.
- The margin is the smallest internal intervention that changes the model output.
- Jacobian Lens concept steering vectors apply the internal interventions.
- Correctly learned facts gained larger margins after knowledge-injection training.
- The models and five question-answering datasets are not named in the excerpt.
From the original article
1 min read at huggingface.co
Large language models (LLMs) are increasingly used in critical domains, where their susceptibility to hallucinations makes reliable and interpretable uncertainty quantification essential. We introduce the concept of mechanistic uncertainty, which characterizes uncertainty through the stability of the internal mechanisms supporting a model's response.
By Aneesh Komanduri, Majid Afshar, Timothy A. Miller · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.