BudgetPix: Compute-Adaptive Tokenization for Pixel-Space Image Diffusion
BudgetPix adapts image-diffusion token budgets to visual complexity, matching larger models at a fraction of compute.
BudgetPix is an adaptive tokenization framework for pixel-space image diffusion that allocates tokens by visual complexity using an entropy-guided quadtree and a multi-scale patch embedder. A scale-aware decoder and flexible training schedule let a single checkpoint run across a range of inference compute budgets. On text-to-image generation it matches MiniT2I-L at 512² and PixelDiT at 1024² using 25% of the original compute. With a MeanFlow backbone, class-conditional images need only 60% of full compute for near-zero quality loss, about a 0.8-point FID increase.
- Entropy-guided quadtree encoder produces variable-length tokens from fixed-size images.
- One checkpoint supports a wide range of inference compute budgets.
- Matches MiniT2I-L and PixelDiT quality at 25% of compute.
- MeanFlow class-conditional results need 60% budget for a 0.8 FID rise.
- Human and VLM judges report a better quality-efficiency tradeoff.
From the original article
1 min read at huggingface.co
Most image generation models rely on uniform tokenization, allocating the exact same computational budget to equally-sized image patches. This static paradigm cannot adapt to different resource constraints at inference time, and yields suboptimal quality-cost tradeoff by devoting the same effort to both plain backgrounds and intricate details.
By Ozgur Kara, Yujia Chen, Daniel Watson · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.