Retrieval Capacity of Self-Attention Under Competition
Researchers measure how few high-attention tokens language models need to match full-context prediction loss.
1 stories, newest first · one card per story
Researchers measure how few high-attention tokens language models need to match full-context prediction loss.
Today 1.1K readers · 1.5K views · 474 reads · all time 184K views