CPU-ARC caches LLM reasoning traces for cheaper CPU reuse
Original title: CPU-ARC: Agentic Reasoning Caching for on-CPU Reasoning Reuse
The paper reports on-par results with frontier models on hard math and science tasks at 16–291x lower cost.
Researchers introduce agentic reasoning caching, which builds CPU-executable caches (ARCs) from LLM reasoning traces so a solved task can be applied to new instances with little language-model involvement. They say ARCs match frontier LLMs on difficult mathematical and scientific tasks, remain explainable and continually trainable, and resist input perturbations. Invocation is reported as 16–291x cheaper than the source model.
Unconfirmed: Performance, robustness, and cost figures are the authors' reported results, not independently verified here.
- ARCs are built from LLM reasoning traces and run on CPU.
- Reported speedup versus the source model is 16–291x.
- Authors say caches stay robust to input perturbations.
- The method is described as continually trainable and fully explainable.
From the original article
1 min read at huggingface.co
Large language models reinvent the wheel all the time. They conflate task-solving with the execution of the solutions, which forces them to solve every problem from scratch. We show that once a task has been reasoned through, the application of its solution to a new problem instance can be done very efficiently and with little involvement from language models.
By Peter Belcak, Greg Heinrich, Saurav Muralidharan · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.