Same Bytes, Different Authority: Reserved-Token Representations in Chat-Template Prompt Injection
Reserved chat-template tokens give injected instructions far more authority than identical subword text on open-weight models.
Researchers show prompt injection against LLM agents is much stronger when forged chat-template markers arrive as reserved control tokens rather than ordinary subwords, despite decoding to the same text. On InjecAgent, subword encoding lowered attack success by 39 to 66 percentage points on three of four open-weight families; Qwen3-8B showed only an 8-point gap unless reasoning was suppressed, widening it to 50. Instruction tuning strengthens preference for reserved markers, and the usual tokenizer mitigation leaves tool-protocol tokens intact in 33 of 67 configurations covering 255 of the 400 most-downloaded Hugging Face chat models.
- Subword encoding of forged markers cuts InjecAgent success by 39-66 points on three families.
- On Qwen3-8B the gap is 8 points, or 50 if reasoning is suppressed.
- Adaptive attackers find non-reserved embedding neighbors on three of four families.
- Special-token mitigation misses tool-protocol tokens in 33 of 67 tokenizer configs.
Full article304 words · extracted from huggingface.co · click to collapse
Prompt injection against LLM agents becomes much stronger when the injected instruction is wrapped in the model's own chat template. A forged template marker such as <|im_start|> can reach the model either as a single reserved control token or as a sequence of ordinary subword tokens. The two decode to exactly the same text, and because tokenization runs on the server, the defender rather than the attacker decides which one the model receives. We use this to measure how much of the injected instruction's authority comes from the reserved token's learned representation. Encoding the forged markers as subwords, with the text held fixed and a control for the extra tokens this adds, lowers attack success on the InjecAgent benchmark by 39 to 66 percentage points on three of four open-weight families, and the gap carries over to multi-turn agent tasks in AgentDojo. On Qwen3-8B the gap is 8 points, because without reserved ids the model still recognises the forged turn from its text by reasoning; suppressing the reasoning block widens the gap to 50. The authority sits in the single learned vector at the marker position: the mean of the marker's subword vectors does not reproduce it, the vector of the nearest ordinary token restores the attack on Llama-3.1, and an adaptive attacker who searches for non-reserved markers finds such embedding neighbours on three of four families. In every base and instruction-tuned pair we test, instruction tuning strengthens the model's preference for reserved markers. The standard mitigation, a tokenizer option that encodes special tokens as ordinary subwords, applies only to tokens a configuration declares special, so in 33 of 67 distinct tokenizer configurations, covering 255 of the 400 most-downloaded chat models on Hugging Face, it leaves intact the tool-protocol tokens through which agents read untrusted tool output, and the gap persists on that channel.
Text extracted automatically; images, tables and formatting may be missing. Original: https://huggingface.co/papers/2609.35932