Zero-valued PyTorch loss can run code in the backward pass
Original title: How to Run Turing-Complete Malware in PyTorch's Backward Pass
IEEE 754 rounding and autodiff disagree, so a loss logged as 0.0 can still set gradients that rewrite weights.
A research write-up shows a PyTorch loss that prints 0.0 yet makes backward() and a learning-rate-1 SGD step replace selected weights with the next state of an arbitrary program, including Conway’s Game of Life. The construction uses float32 truncation around 2^30, which hardware treats as zero while symbolic differentiation treats it as identity. The author frames it as a fine-tuning supply-chain trick: a regularizer never mutates weights in the forward pass and leaves the base checkpoint hash unchanged.
Unconfirmed: The post is a research demonstration and does not report real-world use of the technique.
- Target gradient is current weights minus the desired next state, divided by the learning rate.
- At exponent 30, float32 steps by 128, so additions under 64 round away.
- The example regularizer is added to a Llama 3 8B training loss and logs 0.000000.
- The attacker does not change the checkpoint, architecture, dataset or optimizer.
Impact: Hidden arbitrary computation and weight changes during training
| Product | Affected |
|---|---|
| PyTorch | — |
From the original article
14 min read at elonlit.com
Originally I was toying around with a question about floating-point precision in PyTorch: as a joke, I was curious to see if it's possible to find some weird loss \(\mathcal{L}\) for which calling loss. backward() will result in some change of weights while loss. item() == 0.
By elonlit.com via rrampage · elonlit.com
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.