ALIVE: Interaction-Aligned Object Insertion for First-Frame-Guided Video Editing
ALIVE inserts objects into videos so they interact coherently, using only an edited first frame.
ALIVE is a video-editing framework that makes inserted objects take part in actions such as being picked up, using an edited first frame and an instruction that names only the added object. The authors curate 35,800 pairs from 3D-rendered, model-generated, and real-world videos plus ROSE editing pairs, and train a vision-language model to predict interaction guidance. Without that guidance, ALIVE improves Overall by 43.9% and 4.4% over the strongest baseline on the ALIVE-interaction and general insertion benchmarks. Predicted guidance adds 0.95 points on ALIVE-interaction without extra user input.
- Training set has 35,800 pairs that preserve surrounding action.
- A VLM predicts interaction guidance from the first frame and instruction.
- Overall rises 43.9% on ALIVE-interaction versus the strongest baseline.
- Guidance adds 0.95 points without further user inputs.
From the original article
1 min read at huggingface.co
Current video editors can insert objects but often struggle to make them participate in interactions such as being picked up or manipulated. We introduce ALIVE, a framework that makes inserted objects "alive" through coherent interactions with the source video's contents, using an edited first frame and an instruction naming only the added object.
By Zhenghong Zhou, Zhe Lin, Jiebo Luo · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.