Towards Looped Models Done Right, Part II: Rethinking at Fixed Points
Looped language models near fixed points cut training and decode cost via learned depth priors and orthogonal injection.
The paper argues that looped language models approaching fixed points need less of the recurrent path, enabling cheaper training and inference. It reports truncated backpropagation, terminal KV sharing with almost no accuracy loss, a distilled student that prefills up to 1.79x faster, and RL updates from saved states that are 2x faster than replaying trajectories. A learned depth prior and orthogonal input injection lower perplexity from 100M to 1.6B parameters versus Huginn's prior and existing injection. At 1.6B, the learned prior with a 3x smaller KV cache matches fixed-depth training's downstream average with the full cache.
- Fixed-point recurrence enables truncated backprop and terminal KV sharing
- Distilled student prefills up to 1.79x faster
- RL updates from saved rollout states are 2x faster
- Learned prior and orthogonal injection lower perplexity from 100M to 1.6B
- 1.6B model matches full-cache scores with a 3x smaller KV cache
From the original article
1 min read at arxiv.orgEvery recurrence of a looped language model adds cost in training, decoding, prefill, and reinforcement learning (RL). The closer recurrent states get to fixed points, the less the path to them matters. This enables truncated backpropagation in training; terminal key-value (KV) sharing for decoding with almost no loss in accuracy; a distilled student that prefills up to 1.
By Benhao Huang · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.