Skip to content
NewA calmer, faster ZeroHour

PixelDense: Dense Prediction as Representation Alignment for Pixel Diffusion

Hugging Face daily papersby Lehan Yang, Daiqing Qi, Wenhao Zhang ()1 min read3 · 1 read
AI summary · grok-4.7Summaries by Vani AI

PixelDense separates semantic and geometric alignment teachers, improving pixel diffusion quality and training speed.

PixelDense uses dense-prediction models as representation-alignment targets for pixel-space diffusion, splitting DINOv2 and SAM2 into a semantic stream and Depth Anything v2 and Metric3D v2 into a geometric stream, with a weight-space orthogonality penalty. A flat sum of all four teachers underperformed the best geometric teacher because semantic and geometric gradients competed. Teachers are frozen during training and dropped at inference. On PixelGen and DeCo, PixelDense improved GenEval, DPG-Bench, and HPS v2.1, lifting PixelGen-XXL GenEval Overall from 0.7927 to 0.8093, with up to 53.1% PQ gain and 36.0% depth AbsRel reduction in partial-noise probes.

Key facts
  • Semantic and geometric teachers use separate projection streams plus an orthogonality penalty.
  • All four teachers stay frozen in training and are removed at inference.
  • PixelGen-XXL GenEval Overall rises from 0.7927 to 0.8093.
  • From scratch, PixelDense hits the baseline peak GenEval 1.23x faster.
  • SDEdit background PSNR on PIE-Bench improves by up to 2.2 dB.
Explain this AI news with Vani AIWhat was announced, what is new and why it matters, in a new chat

From the original article

1 min read at huggingface.co

Representation alignment (REPA) accelerates diffusion transformer training, but its alignment targets are almost exclusively semantic encoders such as DINOv2 and CLIP. Recent analysis points to spatial structure, not global semantics, as the carrier of the alignment effect, yet dense-prediction foundation models trained to predict that structure remain overlooked as REPA targets.

Hugging Face daily papers

By Lehan Yang, Daiqing Qi, Wenhao Zhang · huggingface.co

Read the full article at Hugging Face daily papers

A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.