The More It Says, the More You Pay: A Black-Box Audit of Provider-Side Token Inflation in LLM Services
Researchers define Provider-Side Token Inflation Attacks that inflate LLM output over 10x and release a single-probe audit detecting them with 85.1% accuracy.
The paper defines the Provider-Side Token Inflation Attack (PTIA), where dishonest LLM providers covertly inflate output tokens while preserving task utility, instantiating five attacks at the query, prompt, representation, and model levels that raise mean output length above 10.2x the clean baseline. The authors trace PTIA saturation to sharply lowered end-of-sequence token probability and build a lightweight single-probe audit requiring no trusted reference model, achieving 85.1% average detection with false positives below 2% across four open-weight models. Audits of 15 real LLM API services flagged 7 for PTIA-consistent behavior.
- PTIA defines five provider-side attacks at query, prompt, representation, and model levels, inflating output over 10.2x.
- Attack saturation is traced to a sharply lowered end-of-sequence token probability after the initial intervention.
- Single-probe audit achieves an 85.1% average detection rate with false positives below 2% on four open-weight models.
- Audit of 15 real LLM API services flags 7 for PTIA-consistent behavior.
Full article224 words · extracted from arxiv.org · click to collapse
In pay-per-token LLM services, the more a model says, the more users pay. Dishonest providers can covertly manipulate generation to inflate output tokens while largely preserving task utility. We define such manipulation as a Provider-Side Token Inflation Attack (PTIA) and instantiate five representative attacks at the query, prompt, representation, and model levels of the provider-controlled pipeline. Our experiments show that each attack increases mean output length to more than 10.2x the clean baseline, demonstrating PTIA's financial appeal and feasibility at multiple stages of generation. Yet auditing PTIA from black-box responses is difficult for users. Our key observation is PTIA saturation: an initial attack sharply lengthens output, but further strengthening or composition has much less effect. We trace this saturation to stopping behavior: an initial PTIA sharply lowers the end-of-sequence token probability, whereas further intervention lowers it only marginally. Building on this insight, we design a lightweight single-probe audit that applies a controlled lengthening intervention. Under PTIA, the probe induces far fewer additional tokens than under normal service. The audit requires neither a trusted local reference model nor historical clean responses, and its separately issued original and probed requests resemble ordinary traffic, making evasion difficult. Across four open-weight models, it achieves an average detection rate of 85.1% with false-positive rates below 2%. Across 15 real LLM API services, the audit flags 7 for PTIA-consistent behavior.
Text extracted automatically; images, tables and formatting may be missing. Original: https://arxiv.org/abs/2609.20370