First Passage Attention selectively erases intermediate context
A 110M-parameter model matched FoX perplexity and perfectly retrieved passkeys at eight times training length with a compressed cache.
Stories per day · 30 days
1 stories, newest first · one card per story
Stories tagged #forgetting-transformer