Unsloth publishes GGUF of NVIDIA Nemotron 3.5 Bolt E4B
Original title: unsloth/NVIDIA-Nemotron-3.5-Bolt-E4B-GGUF — new model trending #30 on Hugging Face
The Q4_K_M build has 8B total parameters, 4B effective, and a 64K-token context.
Unsloth's Hugging Face page offers a GGUF quantization of NVIDIA-Nemotron-3.5-Bolt-E4B-4GB, listed as trending. NVIDIA further post-trained the small model from Gemma-4-E4B-it for local voice and device assistants, tool use, and instruction following. The card specifies Q4_K_M precision, text input and output, llama.cpp deployment, and the OpenMDW-1.1 license. A companion DFlash drafter is recommended for speculative decoding.
- Input and output context are each up to 64K tokens.
- Post-training data has a cutoff date of May 2026.
- Recommended sampling is temperature 1.0 and top_p 0.95.
- The full-precision BF16 version is published separately by NVIDIA.
- Chat-template controls enable thinking and conversational modes.
- Products
- llama.cpp
From the original article
10 min read at huggingface.cotext-generation · 71,929 downloads · 141 likes · license other
By unsloth · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.