Hybrid wave-attention beats tiny transformers on perplexity, not memory
A 10.94M hybrid scores 5.95 perplexity, but its peak VRAM matches the pure transformer.
Stories per day · 30 days
1 stories, newest first · one card per story
1 stories, newest first · one card per story
A 10.94M hybrid scores 5.95 perplexity, but its peak VRAM matches the pure transformer.
Today 1.5K readers · 2.4K views · 757 reads · all time 189K views