SpecSlot raises pixel-space detail without longer global attention
Original title: Read, Allocate, Route: Role-Coupled Tokenization for Pixel-Space Generative Transformers
On ImageNet-512, SpecSlot improves FID by 2.84 versus a FLOPs-matched JiT control and uses 3.9% fewer FLOPs.
SpecSlot is a pixel-space rectified-flow Transformer that splits each spatial anchor into slots tied to disjoint DCT coefficient bands. Only the lowest-frequency slot joins global attention; other slots interact locally, and a spatial head adds a local correction while state, target and loss stay in pixel space. On ImageNet-256 it matches a FLOPs-matched JiT control's final FID and converges faster. On ImageNet-512 it improves FID by 2.84 with 3.9% fewer FLOPs and is within 0.69 FID of JiT-B/16 at 38.2% lower cost; on text-to-image it beats JiT-B/8 on DPG-Bench by 1.06 at 40.6% lower forward cost.
- Only the lowest-frequency slot from each anchor uses global attention.
- ImageNet-512 result is within 0.69 FID of JiT-B/16 at 38.2% lower cost.
- Text-to-image SpecSlot beats JiT-B/8 on DPG-Bench by 1.06.
- Forward cost on that text-to-image comparison is 40.6% lower.
From the original article
1 min read at arxiv.orgPixel-space generative Transformers typically use one token per image patch, so capturing finer detail requires more tokens and makes global attention increasingly expensive. We observe that finer detail does not necessarily require more global communication: low-frequency structure benefits from global exchange, while high-frequency detail can be modeled locally.
By Jiongfan Yang · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.