Secure Speculative Decoding for Large Language Models
Lossy speculative decoding sharply raises LLM jailbreak and prompt-injection success; SecureSD tightens early-token checks.
The paper studies the security of speculative decoding, where a smaller draft model proposes tokens that a target LLM accepts or rejects. Across lossy methods, efficiency gains increase jailbreak and prompt-injection success much faster than utility degrades, and the loss is traced mainly to early draft tokens. SecureSD applies a stricter verification criterion at early positions and, in security and utility benchmarks, improves security while preserving efficiency and utility versus existing methods.
- First systematic security study of speculative decoding
- Jailbreak and prompt-injection success rise faster than utility falls
- Degradation traced mainly to early draft-model tokens
- SecureSD applies stricter verification at early decoding positions
From the original article
1 min read at arxiv.orgSpeculative decoding accelerates inference for a large language model (LLM), referred to as the \emph{target model}, by first using a smaller model, referred to as the \emph{draft model}, to generate candidate tokens and then verifying them with the target model for acceptance or rejection. Prior studies primarily focused on the efficiency-utility trade-off of speculative decoding, e. g.
By Yichi Zhang · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.