KronLSTM adds a Kronecker Delta Rule for long-context attention
The open-source architecture revises associative memory along one matrix direction and often beats DeltaNet-family models on long context.
Stories per day · 30 days
3 RSS
4 stories, newest first · one card per story
Stories tagged #associative-memory
Bias-Induced Crossover in Absolute Capacity of Dense Associative Memory
Analysis shows biased patterns cut dense associative memory capacity from N^(n-1)/ln N to O(N^(n/2)), with a bias-induced crossover.
Kalman Delta Networks: Uncertainty-aware Associative Memory
Kalman Delta Networks add uncertainty tracking to linear-attention associative memory, improving perplexity and downstream accuracy at 750M and 1.3B scales.
Kalman Delta Networks: Uncertainty-aware Associative Memory
Researchers propose Kalman Delta Networks, adding Kalman-filter uncertainty tracking to delta-rule linear attention, improving perplexity and accuracy at 750M and 1.3B parameters.