The More It Says, the More You Pay: A Black-Box Audit of Provider-Side Token Inflation in LLM Services
Researchers define Provider-Side Token Inflation Attacks that inflate LLM output over 10x and release a single-probe audit detecting them with 85.1% accuracy.
The paper defines the Provider-Side Token Inflation Attack (PTIA), where dishonest LLM providers covertly inflate output tokens while preserving task utility, instantiating five attacks at the query, prompt, representation, and model levels that raise mean output length above 10.2x the clean baseline. The authors trace PTIA saturation to sharply lowered end-of-sequence token probability and build a lightweight single-probe audit requiring no trusted reference model, achieving 85.1% average detection with false positives below 2% across four open-weight models. Audits of 15 real LLM API services flagged 7 for PTIA-consistent behavior.
- PTIA defines five provider-side attacks at query, prompt, representation, and model levels, inflating output over 10.2x.
- Attack saturation is traced to a sharply lowered end-of-sequence token probability after the initial intervention.
- Single-probe audit achieves an 85.1% average detection rate with false positives below 2% on four open-weight models.
- Audit of 15 real LLM API services flags 7 for PTIA-consistent behavior.
From the original article
1 min read at arxiv.orgIn pay-per-token LLM services, the more a model says, the more users pay. Dishonest providers can covertly manipulate generation to inflate output tokens while largely preserving task utility. We define such manipulation as a Provider-Side Token Inflation Attack (PTIA) and instantiate five representative attacks at the query, prompt, representation, and model levels of the provider-controlled pipeline.
By Leilei Chen · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.