BeNice! trains looped models to share one KV-cache across recurrences
In simulated low-memory long-context inference, cache hits rose up to 31.4 percent and throughput up to 23.6 percent.
Stories per day · 30 days
Relatedhuginn2kv-cache2language-models2architecture1benice1efficiency1fixed-points1inference1kv-sharing1length-generalization1long-context1mixture-of-experts1qwen1recurrence1state-tracking1
4 stories, newest first · one card per story
Stories tagged #looped-models
PAR aligns looped model steps for longer state tracking
On swap-S5, a Qwen2.5-0.5B model trained on at most 32 moves reaches 95.5% full-state accuracy at 512 moves.
Huginn-style looped models beat Ouro at matched scale
A Huginn-style MoE uses 75% fewer resident parameters than a 112-layer feedforward baseline under matched FLOPs.
Towards Looped Models Done Right, Part II: Rethinking at Fixed Points
Learned depth priors and orthogonal injection improve looped language models while enabling cheaper KV sharing.