Gen4P: Video Generators are 4D Perceivers
Gen4P turns a pretrained video generator into a single-pass 4D perceiver for tracking, depth, and pose.
Gen4P is a unified framework that recovers 4D geometry and motion from monocular video by repurposing a pretrained video generator without iterative denoising. Clean video latents are processed in one encoder pass, and a lightweight cross-attention decoder answers on-demand spatial and continuous-time queries. A frozen pretrained video-depth LoRA is used as a geometry prior without adding trainable parameters. The authors report state-of-the-art or competitive results on 3D point tracking, video depth, and camera pose estimation.
- Repurposes a pretrained video generator as a feed-forward 4D perceiver.
- Predicts 4D attributes from clean latents in one encoder pass.
- A frozen video-depth LoRA adds geometry without extra trainable parameters.
- Reports state-of-the-art or competitive tracking, depth, and pose results.
From the original article
1 min read at huggingface.co
We introduce Gen4P, a unified framework for recovering 4D geometry and motion from monocular video. We show that a pretrained video generator can be repurposed into a feed-forward 4D perceiver without iterative denoising. Building on on-demand querying, Gen4P processes clean video latents in a single encoder pass and predicts the requested 4D attributes through a lightweight cross-attention decoder.
By Yitong Xia, Zeren Jiang, Chuanxia Zheng · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.