AV-FABLE benchmarks sparse audio-visual manipulations in long videos
Original title: AV-FABLE: Audio-Visual Manipulation Forensics in Long-Form Videos
Zero-shot audio-visual language models are near chance, while separated readouts of frozen features lead localization.
AV-FABLE is a dual-timeline benchmark for brief AI edits inside otherwise authentic long-form audio-visual video. It contains 16,650 pristine–manipulated twin pairs, 998 hours, built with 27 editing configurations and separate audio and visual timelines. Zero-shot prompting of audio-visual large language models yields near-chance detection and poor temporal localization. NFSR, which reads frozen features with separate audio and visual decoders, leads in-domain and on three out-of-distribution settings. The benchmark, code, and weights will be released.
- Existing audio-visual deepfake benchmarks mostly use short talking-head clips.
- Separate timelines expose errors that joint-timeline evaluation hides.
- Supervised readouts of frozen perception features recover strong forensic signals.
- NFSR also transfers to held-out videos from an external long-video benchmark.
From the original article
1 min read at huggingface.co
Localized AI manipulations in long-form videos often affect brief events within otherwise authentic content, making video-level classification insufficient for forensic analysis. Existing audio-visual deepfake benchmarks largely focus on short talking-head clips, while long-form temporal forensics remains predominantly visual.
By Yuesheng Huang, Hoi Fan Au, Jiaqi Hou · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.