StreamingSwap aims at real-time identity-preserving video face swapping
Original title: From Short Clips to Real-Time Streams: Identity-Aware Supervision and Context-Efficient Generation for Video Face Swapping
Authors report competitive identity transfer at 20 FPS on one RTX 4090, stable on videos up to 10 minutes.
StreamingSwap is a reference-guided flow-matching framework for streaming video face swapping that transfers a reference identity while keeping other attributes and motion. Joint Rec-ID Loss balances identity supervision and latent reconstruction by noise level. Face-RoI Generation and Short-Context Rollout Training reduce spatial tokens and temporal context. The authors also introduce HumanFaceVid, with 1,000 hours of cinematic training data and 200 short plus 200 long evaluation videos.
Unconfirmed: Speed, fidelity, and identity-transfer results are reported by the authors only.
- The method targets real-time generation over long video streams.
- HumanFaceVid contains 1,000 hours of cinematic training video.
- Evaluation sets include 200 short and 200 long videos.
- Reported speed is 20 FPS on a single NVIDIA RTX 4090.
- Authors say generation stayed stable on videos up to 10 minutes.
- Vendors
- NVIDIA
- Products
- StreamingSwapRTX 4090
From the original article
1 min read at huggingface.co
Video face swapping transfers the reference identity to a target video while preserving its other attributes and motion. Despite recent progress in reference-guided flow-based methods, jointly achieving faithful identity transfer, high fidelity, and real-time generation over long video streams remains challenging.
By Zekai Luo, Yuwei Liang, Ruiqi Ma · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.