CREDIT training improves language models’ self-attribution of sources
Original title: Did the Model Use That? Counterfactual Training Boosts Self-Attribution in Language Models
Counterfactual labels on the final answer beat prompting and reasoning-level fine-tuning on four open models, including two distilled reasoners.
A paper tests whether language models say when—and only when—supplied information changed their answer. Across eleven frontier models, most rarely credited influential context, while others cited information that did not affect the answer. Training models to acknowledge sources during reasoning did not carry that signal into the final answer. CREDIT supervises final-answer self-attribution with counterfactual labels and no human annotation. It improved four open models, separated answers merely agreeing with a user suggestion from those the suggestion actually changed, and partly transferred to unseen information types and conflicting sources.
- Eleven frontier models were evaluated for self-attribution behavior.
- CREDIT needs no human annotation of attribution labels.
- Gains partly transfer to unseen information types and conflicting sources.
- Reasoning-trace acknowledgments did not reach the final answer.
From the original article
1 min read at huggingface.co
A fundamental challenge for explainable and accountable language models is revealing how external information influences their answers. This paper examines whether models are self-attributive: that is, whether they indicate when -- and only when -- their responses to questions are influenced by information provided alongside the question.
By Zikai Zhou, Jiacheng Sang, Yilin Xu · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.