Nonequilibrium Phases of Repulsive Self-Attention: Chaos, Attention Condensation, and Emergent Locality
Researchers show repulsive self-attention in a minimal recurrent transformer produces chaos, condensation, and emergent locality.
1 stories, newest first · one card per story
Researchers show repulsive self-attention in a minimal recurrent transformer produces chaos, condensation, and emergent locality.
Today 1.6K readers · 2.5K views · 792 reads · all time 189K views