LiquidAI publishes GGUF weights for LFM3-1.7B
Original title: LiquidAI/LFM3-1.7B-GGUF — new model trending #29 on Hugging Face
The repo adds a QAD Q4_0 file meant to recover accuracy lost to Q4_0 quantization.
LiquidAI released LFM3-1.7B-GGUF, quantized weights of LFM3-1.7B for local CPU and GPU inference. The card points users to llama.cpp v0.8.0 or newer, LM Studio with llama.cpp engine 2.65.1 or newer, and Ollama v0.40.12 or newer on macOS. Recommended sampling is temperature 0.1, top_k 50 and repeat penalty 1.1, with different settings for math and science. The Q4_0 file in the repo is a Quantization-Aware Distillation checkpoint trained to recover accuracy lost to Q4_0 quantization.
- Architecture details are on the original LFM3-1.7B model card.
- llama.cpp support starts at build b11960, also called v0.8.0.
- Ollama is invoked as ollama run lfm3 and needs v0.40.12 or newer.
- The QAD file is named LFM3-1.7B-QAD-Q4_0.gguf.
- Math and science sampling uses temperature 1.0, top_p 0.95 and top_k 20.
- Vendors
- LiquidAI
- Products
- LFM3-1.7B-GGUF
- AI models
- LFM3-1.7B
From the original article
2 min read at huggingface.cotext-generation · 97,184 downloads · 142 likes · license other
By LiquidAI · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.