SpecKCast forecasts diffusion features in the frequency domain
Original title: SpecKCast: Spectral Kalman Forecasting for Diffusion Acceleration
A training-free Kalman filter on DCT coefficients skips transformer steps, with up to 6x speedup and under 0.05 GiB extra memory.
SpecKCast is a training-free method that forecasts Diffusion Transformer outputs in the frequency domain so more denoising steps can be skipped. It applies a 2D discrete cosine transform and tracks each coefficient with a lightweight Kalman filter along a band-specific time coordinate from the noise schedule. The transformer runs only when forecast uncertainty exceeds a threshold. On FLUX.1-dev, Qwen-Image, Wan2.1-14B, and HunyuanVideo, the paper reports a better speed-fidelity trade-off than prior caching and forecasting methods.
- Experiments cover FLUX.1-dev, Qwen-Image, Wan2.1-14B, and HunyuanVideo.
- Reported peak acceleration is 6x with less than 0.05 GiB added memory.
- The transformer is evaluated only when forecast uncertainty exceeds a threshold.
- Equal step intervals are treated as nonuniform change via the noise schedule.
From the original article
1 min read at huggingface.co
Diffusion Transformers (DiTs) achieve high-fidelity image and video generation, but they suffer from substantial inference costs due to repeated transformer evaluations across many denoising steps. Feature caching reduces this cost by reusing features across denoising steps. Recent forecasting methods extend this idea by extrapolating features to skipped steps instead of reusing them.
By Javad Rajabi, Hakki Can Karaimer, David B. Lindell · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.