RenderFormer-V2: Neural Rendering with Heterogeneous Scene Primitives
RenderFormer-V2 is a transformer-based neural renderer handling caustics, volumetric scattering and out-of-distribution materials without per-scene training or specialized code.
RenderFormer-V2 is a learned transformer-based neural rendering model that models global light transport as a sequence-to-sequence transformation, handling caustics, volumetric scattering, environment lighting, textured and displaced surfaces and out-of-distribution materials. It uses a two-stage process: a view-independent stage resolving primitive-to-primitive transport, and a view-dependent stage converting the neural scene representation into pixels. Improvements include combined windowed-attention with a rendering-informed attention sink for scalability, support for heterogeneous primitives like environment maps and participating media, and a surface-reflectance-independent neural material encoding, validated across diverse scenes with extensive ablations.