Policy-Mask picks diffusion features under a channel budget
Original title: Policy-Mask: Budget-Aware Adaptation of Diffusion Features to Discriminative Tasks
Authors report 38.9 mIoU on ADE20K and adapt the method to PixArt-Σ and SD3.5 MMDiT.
Policy-Mask is a budget-conditioned policy that jointly chooses timestep, block, and channel subset when extracting features from a frozen diffusion model for discriminative tasks. Normalizing the channel score by subset size keeps the three policy heads on comparable scales. At matched budgets, the authors report gains over fixed extraction and feature-filter baselines on segmentation, classification, and semantic correspondence, with the best channel-budget score at 256 retained channels.
- ADE20K reaches 38.9 mIoU in the authors' evaluation.
- CIFAR-100 hits 75.4% top-1 at 128 dimensions.
- SPair-71k scores 76.3 PCK@0.1.
- Block selection materially changes downstream accuracy in ablations.
- U-Net channels become image-token hidden dimensions for diffusion transformers.
From the original article
1 min read at huggingface.co
Selecting intermediate features from a frozen diffusion model for discriminative tasks is non-trivial because the useful signal is distributed across the denoising trajectory, network depth, and channels. In practice, this requires choosing a timestep, a block, and a channel subset.
By Ivan Listopadov, Maksim Nakhodnov, Aibek Alanov · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.