Kalman Delta Networks: Uncertainty-aware Associative Memory
Kalman Delta Networks add uncertainty tracking to linear-attention associative memory, improving perplexity and downstream accuracy at 750M and 1.3B scales.
32
Kalman Delta Networks add uncertainty tracking to linear-attention associative memory, improving perplexity and downstream accuracy at 750M and 1.3B scales.
Researchers propose Kalman Delta Networks, adding Kalman-filter uncertainty tracking to delta-rule linear attention, improving perplexity and accuracy at 750M and 1.3B parameters.
Entities are extracted by the model from each article. First seen , last mentioned . Watching keeps it in this browser only (no account); the watchlist page and dashboard alerts use it.
Today 1.6K readers · 2.5K views · 779 reads · all time 189K views