RidgeFill speeds prefill with a training-free late-layer cache
Original title: RidgeFill: Training-Free Late-Layer Cache Fill for Faster Prefill
A closed-form ridge map fills upper-layer cache; reported TTFT gains are 1.25–1.60x on tested Qwen setups.
RidgeFill skips computing most upper layers during prefill by predicting later-layer inputs from a split-layer state with a ridge regression fitted once on calibration text. Nothing is trained and no tokens are dropped; a chosen tail and every decode step still run the full model. Measurements on Llama and Qwen models attribute error to keys, values or recurrent state depending on the model. Reported speedups come with small benchmark drops whose confidence intervals often include zero.
Unconfirmed: 95% intervals include zero and do not rule out losses of a few benchmark points.
- On Qwen3.8-27B, TTFT was 1.25 to 1.31x faster at 16K to 120K tokens.
- On Qwen3.8-Flash-Next in vLLM, speedup was 1.50 to 1.60x at 9K to 32K.
- RULER needle tasks were 99.8 versus 100.0 on that Flash run.
- Fitted maps gained 49.1 and 49.6 RULER points versus identity fill on two dense models.
From the original article
1 min read at huggingface.co
RidgeFill shortens prefill by not computing most of a model's upper layers for the prompt. The lower layers run exactly up to a split; from the split layer's state a per-layer map predicts every later layer's input, and each late layer writes its cache from that prediction. The design follows YOCO and DeepSeek-V4.
By Dylan Johnston, Robert P. Smith · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.