To Copy or Not to Copy: Controlling Speculative Decoding via Intrinsic Model Signals
SwitchSD uses internal LLM signals to adaptively switch between neural drafting and context-based copying in speculative decoding, boosting throughput up to 15%.
25
SwitchSD uses internal LLM signals to adaptively switch between neural drafting and context-based copying in speculative decoding, boosting throughput up to 15%.
Today 1.6K readers · 2.5K views · 779 reads · all time 189K views