A Princeton Researcher Proposes Recurrent Looped Transformer (RLT) that Carries Decoder State across Every Token, Fixing 96 Blocks per Token with Unbounded Temporal Depth
Princeton researcher Yifan Zhang proposes Recurrent Looped Transformer, carrying full decoder state across every token for unbounded temporal depth.
Yifan Zhang's technical report defines the Recurrent Looped Transformer (RLT), pairing a causal encoder with a recurrent decoder whose final output and layerwise sliding-window attention cache carry into every subsequent token with no prompt-response boundary reset. The reference configuration ties 48 encoder and 48 decoder layers, executing 96 logical blocks per token while the state path grows to 48t blocks after t tokens at fixed per-token compute. The report details RL replay contracts that rebuild all states under current parameters and exact prefix snapshots for multi-turn serving, but explicitly reports no measured efficiency, reasoning quality, or scaling results.
- Decoder state and SWA cache persist across all prompt and response tokens with no boundary reset.
- Reference config ties 48 encoder/decoder layers; state path reaches 48t blocks after t tokens.
- RL replay rebuilds encoder memory, recurrent output, and SWA caches under current parameters.
- No measured efficiency, reasoning, or scaling results; validation remains open.
From the original article
4 min read at marktechpost.com
Yifan Zhang's Recurrent Looped Transformer (RLT) technical report proposes a causal encoder paired with a recurrent decoder that carries its final hidden state and layerwise sliding-window attention cache across every prompt and response token, with no reset at the serving boundary.
By Asif Razzaq · marktechpost.com
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.