Attention competition predicts context rot in retrieval
Filler tokens maintain accuracy up to 32K tokens in retrieval but rival statements multiply loss by 9-110 across five open models.
Stories per day · 30 days
1 stories, newest first · one card per story
Stories tagged #attention-mechanisms