Muir improves linear attention with gating and readout
Muir reduces perplexity by 1.8-2.0% over GDN-2 and supports 10M-token extrapolation.
Stories per day · 30 days
1 stories, newest first · one card per story
1 stories, newest first · one card per story
Muir reduces perplexity by 1.8-2.0% over GDN-2 and supports 10M-token extrapolation.
Today 1.5K readers · 2.4K views · 757 reads · all time 189K views