Ariadne's Thread of LipSync: Unraveling Forgeries via Inconsistency between Lip Motions and Head Poses
LipDA detects and attributes LipSync forgeries from mismatches between lip motion and head pose.
LipDA is a framework for detecting and attributing LipSync video forgeries by exploiting inconsistency between lip motion and head pose in natural speech. Detection contrasts lip and pose features from authentic and forged videos, while attribution captures generator-specific temporal and audio-visual synchronization patterns. Across two existing datasets and the authors' large multi-generator LipSync-A dataset, it reports more than 97% detection AUC and 97.5% attribution accuracy. Code and the dataset are released on GitHub.
- LipDA contrasts lip-motion and head-pose features from real and forged video.
- Attribution uses temporal dynamics and audio-visual synchronization as model fingerprints.
- Detection AUC exceeds 97 percent on the evaluated datasets.
- Model attribution accuracy reaches 97.5 percent.
- Code and the multi-generator LipSync-A dataset are publicly released.
From the original article
1 min read at arxiv.orgRecent advances in LipSync generation technology have led to the creation of highly realistic videos, posing severe societal risks. However, existing defense strategies struggle against LipSync forgeries, as advanced LipSync generation methods not only achieve better lip synchronization but also eliminate visual artifacts.
By Tianyi She · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.