Skip to content
NewA calmer, faster ZeroHour
StoryAI researchinfo

SpecSlot paper limits global attention in pixel-space transformers

2 sources · 2 articlesFirst reported Updated ()1 · 1 read
What's new

First merged summary.

Merged summary · x-ai/grok-4.7 · rewritten as coverage arrivesSummaries by Vani AI

SpecSlot, a pixel-space rectified-flow Transformer, matches or beats JiT controls at lower cost by routing only low-frequency slots globally.

SpecSlot is a pixel-space rectified-flow Transformer that separates output detail from the length of the global attention sequence. Each spatial anchor expands into slots assigned disjoint DCT coefficient bands; only the lowest-frequency slot joins global attention, other slots interact locally, and a spatial head adds a local correction. Generative state, target, and loss stay in pixel space. On ImageNet-256 the authors report FID comparable to a FLOPs-matched JiT control with faster convergence. On ImageNet-512 they report a 2.84 FID improvement at 3.9% fewer FLOPs and within 0.69 FID of JiT-B/16 at 38.2% lower cost; on text-to-image SpecSlot beats JiT-B/8 on DPG-Bench by 1.06 at 40.6% lower forward cost.

Key facts
  • Only the lowest-frequency slot from each spatial anchor uses global attention.
  • ImageNet-512 FID improves by 2.84 versus a FLOPs-matched JiT control.
  • That ImageNet-512 run uses 3.9% fewer FLOPs than the control.
  • Result is within 0.69 FID of JiT-B/16 at 38.2% lower cost.
  • Text-to-image DPG-Bench exceeds JiT-B/8 by 1.06 at 40.6% lower cost.
  • ImageNet-256 matches the control's final FID and converges faster.
Explain this AI news with Vani AIWhat was announced, what is new and why it matters, in a new chat

Timeline

dated events in the reporting · oldest first
  1. Oct 28, 2026Hugging Face daily papers lists the SpecSlot paper. · Hugging Face daily papers
  2. Oct 29, 2026arXiv posts the SpecSlot paper with the reported results. · arXiv

What each outlet adds

beyond the shared facts
  • Hugging Face daily papersStates ImageNet-256 matches control FID and converges faster.
  • arXivPairs the 2.84 ImageNet-512 FID gain with 3.9% fewer FLOPs.
AI models
SpecSlotJiT

Coverage

oldest first · one row per article · UTC
  1. · 4d ago Hugging Face daily papersFirst report

    SpecSlot cuts global attention cost in pixel-space transformers

    Only the lowest-frequency slot per spatial anchor attends globally; higher-frequency detail stays local.

  2. · 3d ago arXiv cs.AI / cs.LG / cs.CL

    SpecSlot raises pixel-space detail without longer global attention

    On ImageNet-512, SpecSlot improves FID by 2.84 versus a FLOPs-matched JiT control and uses 3.9% fewer FLOPs.