Zero-valued PyTorch loss can run code in the backward pass
IEEE 754 rounding and autodiff disagree, so a loss logged as 0.0 can still set gradients that rewrite weights.
Stories per day · 30 days
3 stories, newest first · one card per story
Stories tagged #floating-point
NeuralZip: Reusable Setup for Fast Lossless Compression
NeuralZip reuses a precomputed exponent setup to losslessly compress model checkpoints up to 21.33 times faster than baselines.
OPEN-1B: A Fully Auditable Training Run
Open-1B releases a 1B-parameter model with bitwise-reproducible training, letting independent auditors verify every step of the run on commodity hardware.