Transformers forget prompts as generated context grows
Original title: A Conditional No-Hair Theorem for Transformers: How Language Models Forget Their Prompt
Hidden-state distance between two prompts decays as a power law; trained models forget faster than untrained ones.
The paper studies how much an autoregressive transformer retains of its prompt after generating t tokens. Hidden-state distance between two prompts followed by the same continuation decays approximately as a power law. Trained models forget faster than untrained ones. A proof bounds the offset for finite-depth pre-LN softmax transformers when pre-norm activations stay away from zero, and a linear relay model shows the bound is tight.
- Trained models show alpha about 0.55 to 0.7; untrained ones about 0.2 to 0.5.
- The bound is O of P times (log t) to the L-1, divided by t.
- Relay through continuation tokens adds only logarithmic factors.
- A logit bias that cancels dilution stops the decay, then reverses it.
From the original article
1 min read at huggingface.co
We ask how much an autoregressive transformer retains of its prompt after generating t tokens.
By Shuyang Xiang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.