Entity-replacement responses transfer across questions
Original title: Cross-Prompt Reuse of Entity-Replacement Responses
Hidden-state reuse removes 45-49% of no-change errors on 0.5-1B Qwen and Llama models for entity-replacement responses.
With fixed weights, entity-replacement responses transfer between questions. Hidden-state change adds aligned change at 50% depth. Removes 45% and 49% of no-change error on 0.5-1B Qwen and Llama. Gemma-2-2B passes comparisons after restoring BOS token. On small checkpoints, it reproduces shared shifts without reliably lowering relation-selective error.
- Hidden reuse removes 45% and 49% of no-change error on Qwen and Llama
- Gemma-2-2B passes comparisons after restoring BOS
- Other-relation sources mainly reproduce shared shifts
- Relation-neutral change lowers relation-selective error more than random changes
- Organizations
- huggingface
- AI models
- qwenllamagemma-2-2b
From the original article
1 min read at huggingface.co
With model weights fixed, we measure how much of an entity-replacement response transfers between English factual questions, beyond whether answers change. We add the hidden-state change from two source questions to a target question about the same entity, comparing the predicted log-probability shift between two relations' published answers, given known answer prefixes, with the measured one.
By Jinkun Chen, Xuemin Yu, Sijia Han · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.