LLM code self-refinement often raises energy versus human solutions
Original title: Green Code or Costly Code? Evaluating the Energy Costs of Iterative LLM-Based Code Optimizations
Of 44 configurations, 42 used more energy per problem, with overheads from 4% to 886%.
Researchers measure whether LLM self-refinement that optimizes code for speed also cuts energy, comparing results with canonical human solutions on all 1,000 EffiBench problems. Eleven open-source models were run in four decoding setups with hardware-level energy metering. Inference energy ranged from 0.79 to 8.68 MJ, and capping output tokens cut energy by up to 69%. Only two configurations beat the human reference, and only marginally.
- Inference energy spans 0.79 to 8.68 MJ across model-variant pairs.
- Token limits reduced energy use by up to 69% versus the default.
- Break-even for the two better cases is about 10^6 to 10^7 runs.
- Mixture-of-Experts used less inference energy than same-size dense models.
- Artifacts are at doi.org/10.5281/zenodo.23598170.
From the original article
1 min read at arxiv.orgLarge Language Models (LLMs) are increasingly used to optimize code for performance. This practice often assumes that improvements in execution speed naturally translate into overall lower energy consumption.
By Sadra Emami · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.