SIMPLE: a four-parameter law predicts LLM inference energy cost from architecture
Original title: The SIMPLE Law for Energy Cost of LLM Inference
Across 13 dense LLMs on H100 GPUs the law reaches R2 of 0.969 and 11.97% MAPE, and shows agentic requests cost 13x more energy than non-agentic ones.
The paper derives closed-form expressions for the computational and memory-access complexity of LLM inference, covering MHA and GQA, and distills them into SIMPLE, a four-parameter law separating prefill, KV-cache and next-token costs. Validated on 13 dense LLMs from six families on NVIDIA H100 GPUs without per-LLM calibration, it reaches R2 of 0.969 and 11.97% MAPE, several times better than the state of the art. The law also explains findings: aggressive GQA grouping cuts per-token energy up to 3x versus MHA, and agentic serving raises energy per request by 13.0x.
- Prefill energy grows quadratically with hidden dimension, so larger models suffer most from long inputs.
- Intelligence per Joule peaks at an intermediate reasoning budget and falls up to 3.14x over a 4.0x budget range.
- Prefix caching removes 93% of prefill cost but leaves KV-cache cost unchanged; cached agentic requests still cost 4.8x more.
From the original article
1 min read at huggingface.co
Generative AI is billed per token, yet tokens are not equally expensive. Each token enters the KV-cache and raises the cost of those that follow, so energy grows non-linearly along a sequence whose tokens look identical. No analytical model states what a token costs, and the most accurate SOTA model is a fit, not a law: it absorbs architecture into fitted coefficients, requiring per-LLM fitting.
By Hiari Pizzini Cavagna, Andrea Proia, Daniele Cesarini · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.