RelightFormer: Feed-forward Generative Transformer for Multiview Object Relighting
RelightFormer is a feed-forward generative transformer for photorealistic single- and multi-view object relighting, trained on a 90K-object dataset.
Researchers introduce RelightFormer, a feed-forward generative transformer adapted from a video foundation model that performs direct image relighting without explicit intrinsic property estimation. The architecture injects target environment maps via a latent illumination module with cross-attention and uses permutation-invariant positional encodings for unordered multi-view inputs. Training relies on the newly constructed Laval Objaverse Dataset (LOD) with 90K objects and 39K unique illuminations, and the model shows state-of-the-art quality with strong zero-shot generalization across single-view, multi-view, and novel-view relighting.
- Bypasses explicit inverse rendering and intrinsic estimation
- Latent illumination module injects environment maps via cross-attention
- Trained on the Laval Objaverse Dataset: 90K objects, 39K illuminations
- State-of-the-art zero-shot relighting across single- and multi-view tasks
- AI models
- RelightFormerLaval Objaverse Dataset
From the original article
1 min read at huggingface.co
Image relighting is traditionally tackled via complex inverse rendering pipelines, which suffer from ill-posed optimization, or single-image generative models that ignore crucial multi-view cues necessary for understanding 3D geometry and material interactions.
By Hejun Wang, Jinxi Li, Junwei Jiang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.