Recipe adapts pretrained autoregressive image models to any token order
Original title: Unlocking Any-Order Generation in Pretrained Autoregressive Image Models
On LlamaGen and RAR, the method beat MaskGIT inpainting FID 6.21 to 10.44 at 13-33% extra compute.
The paper says fixed raster-scan training blocks inpainting, outpainting, and editing, while training any-order models from scratch costs about 3x standard compute and discrete diffusion drops AR inference compatibility. The recipe adds future-aware positional embeddings and an ordering curriculum to pretrained raster-scan models. On LlamaGen-L/XL and RAR-L/XL it beat MaskGIT on ImageNet inpainting (FID 6.21 versus 10.44) and outpainting, stayed competitive with MAR-L, and raised class-conditional FID by 1.2-1.9 while keeping KV caching and vLLM usable. Code is at Ordered-AR/unlockedadaptation.
- Extra compute beyond pretraining was 13-33%.
- Class-conditional FID rose by a factor of 1.2-1.9.
- The standard AR inference stack, including KV caching and vLLM, is left unchanged.
From the original article
1 min read at huggingface.co
Autoregressive (AR) image generators are powerful in principle but inflexible in practice: training with a fixed raster-scan order hardcodes a single factorization of the image distribution into the model, leaving it unable to inpaint, outpaint, or edit at inference time.
By Rishav Pramanik, Marco Pedersoli, Zhaozheng Yin · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.