Dory1.0-4B looped language model released
Original title: Dory1.0-4B: A Looped Normalized-GPT
4-billion-parameter looped model outperforms larger models on benchmarks with open checkpoints.
Dory1.0-4B uses a normalized Transformer backbone with elastic depth for variable inference compute. Pre-trained on 6 trillion tokens then post-trained with SFT and RLVR. With multiple loops it exceeds Qwen3.5-9B and Gemma models on average across benchmarks.
- 4B parameters with looped architecture
- Outperforms 9B and 26B models
- Open-source BF16, MXFP8, NVFP4, GGUF checkpoints
- Pre-trained on 6 trillion tokens
- KV-cache equals non-looped model size
- AI models
- Dory1.0-4B
From the original article
1 min read at huggingface.co
We introduce Dory1. 0-4B, a 4-billion-parameter looped (latent-recurrent) language model with elastic depth: a single checkpoint serves different inference compute budgets. Dory1.
By Maxim Fishman, Simeng Sun, Shantanu Acharya · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.