Transformers forget prompts as generated context grows
Hidden-state distance between two prompts decays as a power law; trained models forget faster than untrained ones.
1 stories, newest first · one card per story
Hidden-state distance between two prompts decays as a power law; trained models forget faster than untrained ones.
Today 1.6K readers · 2.5K views · 779 reads · all time 189K views