NVIDIA releases Dory1.0-4B, a looped 4B reasoning model
Original title: nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-BF16 — new model trending #30 on Hugging Face
The BF16 checkpoint uses a 256K context, OpenMDW-1.1, and variable loop counts without retraining.
NVIDIA published NVIDIA-Nemotron-Labs-Dory1.0-4B-BF16 on October 16, 2026, a 4-billion-parameter English model for general-purpose reasoning. It uses a looped normalized-GPT with interleaved sliding-window and full attention, applying a shared 20-layer recurrent block between 8-layer input and output blocks. Changing the loop count trades inference compute for accuracy from one checkpoint. Weights are also listed in MXFP8, NVFP4, and Q5_K_M, under OpenMDW-1.1, for commercial or non-commercial use.
Unconfirmed: The model card is truncated, so later benchmark rows are not available here.
- Last-loop KV caching avoids extra cache growth as loop count rises.
- The model was pre-trained natively in NVFP4, then continued in BF16.
- Four loops score 61.8 on SWE-bench Verified versus 50.3 at one loop.
- Listed comparisons include Qwen3.5-4B, Qwen3.5-9B, and Gemma 4 26B A4B.
- Vendors
- NVIDIA
From the original article
9 min read at huggingface.cotext-generation · 2,480 downloads · 145 likes · license other
By nvidia · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.