MarkSec: Capability-Aware Evaluation of Adversarial Attacks Against LLM Watermarks
MarkSec unifies evaluation of stealing, scrubbing, and spoofing attacks against LLM watermarks with quality-constrained success metrics under shared reporting protocols.
MarkSec is a framework unifying analysis of stealing, scrubbing, and spoofing attacks against LLM watermarks under shared detector calibration, metric definitions, and reporting protocols. It introduces a quality-constrained attack success metric that jointly assesses attack effectiveness and text quality. Experiments across representative watermark families, attacks, LLMs, and datasets show that attacks strongest by watermark removal alone can fall behind general rewriting when success requires acceptable text quality, and stealing-based scrubbers often underperform the best general-scrubbing baselines.
- Unifies stealing, scrubbing, and spoofing watermark attacks under a common protocol
- Quality-constrained success metric jointly measures attack effectiveness and text quality
- General rewriting remains a strong scrubbing baseline across watermark families
- Stealing-based scrubbers often underperform general rewriting when quality is required
From the original article
1 min read at arxiv.orgLLM watermarking helps trace the origin of generated text, but faces stealing attacks that recover watermark information, scrubbing attacks that remove watermark signals, and spoofing attacks that forge text accepted as watermarked. These attacks are often studied in isolation, leaving their connections unclear. Evaluations also often lack shared detector calibration, metric definitions, and reporting protocols.
By Kairong Li · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.