Bonsai 2 27B: Near-Lossless Compression in a 9x Smaller Footprint
Prism ML's Ternary Bonsai 2 27B GGUFs require a custom llama.cpp fork to run.
Simon Willison says Hugging Face GGUFs for Prism ML's Ternary-Bonsai-2-27B, billed as near-lossless compression in about a 9x smaller footprint, do not run on stock llama.cpp. The weights need Prism's llama.cpp fork, including its macOS arm64 runtime, to load the roughly 5.95 GB GGUF. The note is setup guidance for an existing open-weights release rather than a new benchmark paper.
- Ternary Bonsai 2 27B claims near-lossless compression at about 9x smaller size.
- The published GGUF is roughly 5.95 GB on Hugging Face.
- Stock llama.cpp cannot load it; Prism's llama.cpp fork is required.
My comment on Bonsai 2 27B: Near-Lossless Compression in a 9x Smaller Footprint — Hacker News. If you want to try out out the GGUFs from https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-gguf#th... be aware that you need Prism's llama.cpp fork to get them to work, from https://github.com/PrismML-Eng/llama.cpp/releases/tag/prism-... This should work: cd /tmp # Get the Prism macOS runtime curl -fL https://github.com/PrismML-Eng/llama.cpp/releases/download/prism-b10685-7dffb15/llama-prism-b10685-7dffb15-bin-macos-arm64.tar.gz -o bonsai-runtime.tar.gz tar -xzf bonsai-runtime.tar.gz # Get the ~5.95 GB GGUF model: curl -fL…
This source does not provide full text. Read it at simonwillison.net.
Hacker News · AI