Necessary or Sufficient? Evaluating LLM Explanations With Behavioural Evidence
New framework tests whether LLM-cited explanation factors are necessary or sufficient, finding weak correlation across Claude, GPT, and Gemini models.
An arXiv paper introduces black-box intervention tests measuring whether factors LLMs cite in their explanations are necessary or sufficient for their outputs in agent oversight workflows. Across eight models from the Claude, GPT, and Gemini families, Spearman correlations between cited rankings and measured influence ranged from 0.349-0.354 (advisor recommendation) to 0.431-0.580 (prompt monitoring). Uncited factors scored above the lowest cited factor in up to 57.6% of advisor responses, showing cited top-three factors do not reliably identify the most influential inputs.
- Framework estimates necessity and sufficiency scores via controlled black-box interventions
- Eight models from Claude, GPT, Gemini families evaluated on two synthetic tasks
- Cited rankings correlate weakly with measured influence (Spearman 0.349-0.580)
- Uncited factors outranked lowest cited factors in over half of advisor cases
From the original article
1 min read at arxiv.orgLLM decision components that can operate within agent workflows often produce action-relevant recommendations or judgements together with explanations. Operators may use the named factors to monitor a system, diagnose errors, or decide when to escalate an output. Such use assumes that the explanations agree with the component's observable decision behaviour.
By Urja Pawar · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.