Certification of Real Images through Calibrated Content Authentication
Deepfake detectors fall from 99.5% to 76% accuracy; calibrated authentication bounds false certification at 1%.
An evaluation of twenty deepfake detectors against ten generators from the last four years found accuracy falling from 99.5% to 76%, and adversarial perturbations reduced every baseline below 2%. The authors argue content alone cannot establish provenance because a generator can faithfully reconstruct authentic images, making authenticity plausibly deniable. Their calibrated detector can be set so at most 1% of known generated content is wrongly certified, including under bounded adaptive attacks that break baselines. Post-hoc checks also weakened: 1,116 of 3,000 Reddit images resisted a 2022 generator, but only 55 to 79 resisted 2024 generators.
- Twenty detectors fell from 99.5% to 76% accuracy on ten recent generators.
- Adversarial perturbations cut every baseline detector below 2% accuracy.
- Calibration limits wrongful certification of generated content to at most 1%.
- Of 3,000 Reddit images, 1,116 resisted a 2022 generator but only 55–79 resisted 2024 models.
- Tags
- deepfakecontent-authenticationgenerative-modelsadversarial-robustnessprovenancemedia-forensicscalibration
- Organizations
From the original article
1 min read at huggingface.co
Generative models can synthesize high-quality inauthentic multimedia content that is already being misused at scale. We evaluate twenty deepfake detectors against ten generators released in the last four years and find accuracy decreasing over time, from near-perfect 99. 5% to 76%.
By Sarim Hashmi, Abdelrahman Elsayed, Mohammed Talha Alam · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.