SIMPLE: a four-parameter law predicts LLM inference energy cost from architecture
Across 13 dense LLMs on H100 GPUs the law reaches R2 of 0.969 and 11.97% MAPE, and shows agentic requests cost 13x more energy than non-agentic ones.
Stories per day · 30 days
2 RSS
4 stories, newest first · one card per story
Stories tagged #gqa
Virtual Head Attention
Virtual Head Attention cuts KV cache versus GQA while improving quality and folding into standard GQA inference.
Joint Subspace Sparse Attention for Efficient Long-Context Decoding
JSSA sparse attention stays near full-KV accuracy and raises 128K decoding throughput up to 6.43×.
Grouped Value Attention: Efficient KV Caching via On-Demand Key Reconstruction
Grouped Value Attention stores grouped values and reconstructs content keys via a learned linear map, cutting KV-cache size about 45-47% versus GQA.