Rowhammer bit-flip in Triton slows or corrupts LLM inference
Original title: Amplifying Latency and Cost of LLM Inference via Rowhammer Attack
LLVMhammer raised single-GPU latency 4-16% on four models and could cut BERTScore to 25%.
Researchers present LLVMhammer, a Rowhammer attack that flips a bit in Triton’s JIT compilation pipeline so the generated GPU kernel stays functionally identical but slower. Because Triton caches the kernel on disk, one successful flip persists across later inference runs. Karnage searches Triton’s shared, target-independent passes for useful flip locations, so the attack is not tied to one GPU vendor. Evaluations covered four LLMs on an NVIDIA RTX 4090 and an AMD RX 7900 XTX. The authors also show bit-flips can corrupt outputs.
Unconfirmed: Extra annual operating cost is described as a possible result of the slowdown at multi-GPU scale, not a measured bill.
- Tested models include Qwen2.5-7B-Instruct, Llama-3.1-8B-Instruct, Mistral-7B-Instruct-v0.3 and Deepseek-llm-7B-chat.
- Latency rose 4-16% for an inference server on a single GPU.
- A corrupted-output case reduced BERTScore to as low as 25%.
- Triton’s target-independent stages are shared across GPU vendors.
Impact: Persistent slower inference or corrupted model outputs
| Product | Affected |
|---|---|
| Triton | — |
- Products
- Triton
- Malware
- LLVMhammerKarnage
From the original article
1 min read at arxiv.orgLarge language models (LLMs) achieve high inference efficiency using optimized GPU kernels generated by just-in-time (JIT) compilation frameworks such as Triton. Triton compiles GPU kernels through an MLIR/LLVM-based pipeline. We investigate whether a Rowhammer-induced bit-flip in the CPU's DRAM can degrade GPU-based LLM inference performance while preserving output correctness.
By Mrityunjay Shukla · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.