How Lossless Is Lossless Speculative Decoding? The Role of Numerical Precision in Orthrus
Reproduction study finds Orthrus speculative-decoding trajectories match the reference model in only ~45% of cases under BF16, but 100% under FP32.
1 stories, newest first · one card per story
Reproduction study finds Orthrus speculative-decoding trajectories match the reference model in only ~45% of cases under BF16, but 100% under FP32.
Today 1.5K readers · 2.4K views · 757 reads · all time 189K views