Cactus-Compute/needle3 — new model trending #30 on Hugging Face
Cactus Compute released Needle 3, a 121M-parameter model shipped as an 8–29 MB file for on-device tool calls, extraction, and embeddings.
Needle 3 is a foundation model from Cactus Compute targeting mobiles, wearables, robots, automotive, and microcontrollers, delivered as a single 8–29 MB file. Built on a Laddered Simple Attention Network with a Monarch Hadamard MLP, engram n-gram memory, and CQ2-bit quantization, the 121M-parameter model performs tool calling, grammar-constrained structured extraction, and text embedding entirely on-device. Cactus claims it beats models 10x its size on mobile tool calls, and LoRA fine-tuning on DroidCall lifts subnetworks 18–36 points, with tuned 4-layer (29M-parameter) subnetworks passing DeepSeek V4 Flash.
- Single 8–29 MB file runs tool calls, extraction, and embeddings on-device
- Every depth from 2 to 20 layers is a deployable subnetwork
- Grammar-constrained decoding guarantees structured output parses
- Fine-tuned subnetworks from 29M parameters reportedly pass DeepSeek V4 Flash
Full article1,009 words · extracted from huggingface.co · click to collapse
A foundation model for mobiles, wearables, robots, smart home, automotive and microcontrollers. The whole model is a single 8-29 MB file, and we trade general chat capacity to beat models 10x its size on mobile tool calls and match 2-3x bigger models on extraction.
Needle does three jobs, all of them on the device:
- **Tool calls**: given the functions your app exposes, Needle picks the right ones and fills every argument from what the user said. Ask for two things and you get two calls in order; ask for something no tool covers and you get an empty list, not a guess.
- **Structured extraction**: declare a shape, hand over messy text, get typed fields back: an invoice, a booking, a notification, a form. The decode grammar guarantees the output parses, and extraction generalises to classification.
- **Text embedding**: the same model returns a vector for a sentence, so an app can search, match and route locally.
## Model
Needle 3 is a Laddered Simple Attention Network, our small-model recipe: a Monarch Hadamard MLP in place of the FFN, GQA attention with causal conv taps, engram n-gram memory read by gather, and multi-lane hyper-connections, trained so that every depth from 2 to 20 layers is a deployable model. Most of its parameters sit in the engram, so the 121M model does the arithmetic of a 50M one. The weights are compressed to CQ2-bit with Cactus Quants; a byte-level grammar compiled from your schemas constrains every token, and every response carries a calibrated confidence score from a learned head. The architecture diagram is on the [release page](https://cactuscompute.com/needle). The repo holds the 20-layer `needle3.cact`, the `needle3.safetensors` checkpoint to fine-tune, and an engine per platform.
## Benchmarks
Tool calling is exact-match accuracy on the full test splits, extraction is field micro-F1 on the full test splits.
The interactive frontier plot, the architecture and the fine-tuning results are at [cactuscompute.com/needle](https://cactuscompute.com/needle).
## Get started
```sh
pip install cactus-needle
```
Try it in the browser at [cactuscompute.com/needle](https://cactuscompute.com/needle); the Python package and the source are on [GitHub](https://github.com/cactus-compute/needle).
```python
import needle
@needle.tool
def get_weather(city: str):
"Get the current weather for a city."