prism-ml/Ternary-Bonsai-2-27B-gguf — new model trending #29 on Hugging Face
Prism ML released Ternary-Bonsai-2-27B, a 27B ternary-weight model derived from Qwen3.8-27B that runs full reasoning in ~5.95 GB GGUF.
Prism ML published Ternary-Bonsai-2-27B, an Apache 2.0 open-weights model derived from Qwen3.8-27B (27.36B parameters, hybrid attention ~75% linear, 262K-token context). Weights are end-to-end ternary {−1, 0, +1} with g128 FP16 scales at 1.72 bits/weight, shrinking the model from ~54 GB FP16 to 5.95 GB (PTQ1_0) or 7.21 GB (PQ2_0) while reportedly retaining 98.2% of FP16 quality (84.78 average across 14 thinking-mode benchmarks). It ships as GGUF for llama.cpp (CUDA, Metal, CPU) with custom ternary kernels, plus an MLX 2-bit build for Apple Silicon and an optional Q8_0 vision tower pack.
- Ternary g128 weights at 1.72 bits/weight cut the 27B model from ~54 GB FP16 to 5.95 GB.
- Retains 98.2% of FP16 quality: 84.78 average across 14 thinking-mode benchmarks.
- 262K-token on-device context enabled by Qwen3.8-27B hybrid attention (~75% linear).
- Two GGUF packings (PTQ1_0, PQ2_0) with custom ternary kernels; MLX build for Apple Silicon.
- Apache 2.0 licensed, with whitepaper, demo repo, and forked llama.cpp/MLX runtimes.
- Organizations
- Prism ML
- AI models
- Ternary-Bonsai-2-27BQwen3.8-27B
From the original article
12 min read at huggingface.cotext-generation · 0 downloads · 150 likes · license apache-2. 0
By prism-ml · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.