Search: “diffusion-transformer”

Smart search ranks by meaning as well as keywords (one row per story, last 45 days).

Disentangling Representation Evolution in Transformers through Directional Decomposition

Researchers decompose transformer updates into parallel and perpendicular components, linking representation geometry to editing robustness, compression diagnosis, and training interventions.

The paper studies transformer representation evolution as functional geometry, decomposing learned updates into parallel and perpendicular components across attention/MLP and value-aggregation spaces. Targeted edits reveal a space-dependent asymmetry: exclude-self value-space parallel manipulation is markedly more robust than residual-space and perpendicular counterparts. Full-aggregate parallel suppression during from-scratch pretraining lowers validation-loss trajectories and improves downstream averages, with the value-space variant strongest. Code is released on GitHub.

Hugging Face daily papers·updated Sep 14, 17:56 UTC · 1d ago·first Sep 13, 20:00 UTC · 2d agoAI research 2 sources

38

Dynin-Robotics: Omnimodal Unified Diffusion Vision-Language-Action Model

Dynin-Robotics unifies action, goal, and dynamics prediction in one omnimodal masked-diffusion VLA model, reaching 78.4% success on Franka Research 3 manipulation tasks.

Built on the Dynin-Omni masked-diffusion backbone, the model represents language, observations, goals, and actions as discrete tokens and is continually pretrained on roughly 1.33 million trajectories from 48 Open X-Embodiment datasets. The shared trajectory interface enables test-time scaling via goal prediction, action-candidate evaluation, and joint action/future-state refinement. It achieves competitive results on LIBERO and zero-shot LIBERO-Plus, 78.4% average success across four Franka Research 3 conditions, and up to 29.2x faster model-side action decoding from a block-parallel implementation.

Hugging Face daily papers·updated Sep 11, 16:49 UTC · 4d ago·first Sep 10, 20:00 UTC · 5d agoAI research 2 sources

42

Diffusion TV: Experiencing Diffusion Models through Tangible, Embodied Interaction

Researchers present Diffusion TV, a CRT-based installation where antenna manipulation lets audiences physically experience diffusion model denoising.

Diffusion TV is an interactive installation built around a modified CRT television where turning the antenna controls the clarity of AI-generated images and sounds, mirroring the denoising process of diffusion models. Three channels present AI-generated animals from the past, present, and future within a temporal and ecological narrative. The authors frame the work as an embodied, non-verbal alternative to explainable AI that highlights intermediate generative states rather than final outputs.

arXiv cs.AI / cs.LG / cs.CL·Sep 4, 17:51 UTC · 11d agoAI research

12

Marigold V2: Revisiting Diffusion Transformers for Monocular Depth Estimation

Marigold V2 adapts diffusion transformers for monocular depth estimation, improving AbsRel 16-26% over the previous best on KITTI and ETH3D.

Huawei's Bayer lab revisits the Marigold approach to repurpose image generation and editing models built on the diffusion transformer (DiT) architecture into monocular depth estimators. The recipes target single-step inference from pretrained multi-step flow-matching models, with remedies including alignment to ground-truth semantic features and a two-stage fine-tuning protocol using a Sinkhorn-based loss. The resulting model produces crisper depth maps that generalize out-of-distribution and also achieves state-of-the-art results on surface normals estimation and intrinsic image decomposition.

Hugging Face daily papers·Sep 7, 20:00 UTC · 8d agoAI research

42

Diffusion Models and Concept Formation

Paper argues diffusion models implicitly form Cobweb-like concept hierarchies, with a basic level emerging at intermediate noise levels.

The authors draw a formal correspondence between diffusion models and Cobweb, a classic incremental concept-hierarchy learner, noting both are hierarchical Bayesian density models with Gaussian prototypes. Modes of the diffusion model's noisy marginals form a hierarchy whose basic level sits at intermediate noise, where class identity commits. The correspondence is tested on MNIST and Fashion-MNIST via mode-finding. Diffusion is reframed as a cognitive model of concept formation.

arXiv cs.AI / cs.LG / cs.CL·Sep 11, 16:43 UTC · 4d agoAI research

15

RenderFormer-V2: Neural Rendering with Heterogeneous Scene Primitives

RenderFormer-V2 is a transformer-based neural renderer handling caustics, volumetric scattering and out-of-distribution materials without per-scene training or specialized code.

RenderFormer-V2 is a learned transformer-based neural rendering model that models global light transport as a sequence-to-sequence transformation, handling caustics, volumetric scattering, environment lighting, textured and displaced surfaces and out-of-distribution materials. It uses a two-stage process: a view-independent stage resolving primitive-to-primitive transport, and a view-dependent stage converting the neural scene representation into pixels. Improvements include combined windowed-attention with a rendering-informed attention sink for scalability, support for heterogeneous primitives like environment maps and participating media, and a surface-reflectance-independent neural material encoding, validated across diverse scenes with extensive ablations.

Hugging Face daily papers·Sep 3, 20:00 UTC · 12d agoAI research

22

DRIFT: Removing Diffusion Watermarks by Deflecting the Generative Trajectory

Introduces DRIFT, a black-box attack removing diffusion watermarks by deflecting generative trajectories, achieving 98-100% success across nine watermarking schemes.

Researchers propose DRIFT, a black-box watermark removal attack combining partial forward diffusion with stochastic reverse resampling to break trajectory-dependent verification. The paper derives information-theoretic and Wasserstein source-dependence bounds and shows the first verifier-rejected rung is least distorted among rejected rungs. Across nine watermarks spanning three paradigms, DRIFT achieves 98-100% attack success with the best image quality among compared attacks, without secret keys, verifier internals, or per-image gradient optimization.

arXiv cs.CR·Sep 8, 03:54 UTC · 8d agoResearch

38

Model-Aware Schedules Improve Generation via Fiberwise Optimal Transport

Researchers introduce model-aware diffusion schedules via fiberwise optimal transport, cutting flow-matching FID on CIFAR-10 by 38.6% at 16 function evaluations.

The paper proposes constructing diffusion and flow-matching sampling schedules from a fiberwise prediction risk defined via optimal transport, combined with coefficient-path kinetic action, yielding a closed-form time allocation. Across DDPM and flow-matching experiments spanning targets, datasets, and architectures, the schedules beat model-agnostic baselines, including a 38.6% relative FID reduction for flow matching on CIFAR-10 at 16 function evaluations. Normalized fiberwise-risk profiles from independently trained models align closely, suggesting empirical universality, and a frozen analytic allocation template retains most of the gains.

arXiv cs.AI / cs.LG / cs.CL·Sep 10, 17:30 UTC · 5d agoAI research

32

Unlocking Lossless Speedups in LLMs via Discrete Diffusion

Uno pairs autoregressive LLMs with lightweight diffusion weights to draw multiple tokens in parallel, delivering up to 3x lossless speedup without a draft model.

The paper introduces diffusion-augmented LLMs: autoregressive weights trained with the standard next-token objective plus lightweight diffusion weights trained via a Diffusion Distillation phase to emit multiple tokens in parallel. Psi-Spec samplers enable lossless acceleration without the separate draft model required by speculative decoding. The 8B Uno model outperforms the 26B open DiffusionGemma and proprietary Mercury 2 on agentic tool use, coding, and long-context reasoning benchmarks, with up to 3x throughput gains over the base model at all evaluated batch sizes. Code and checkpoints are released publicly.

Hugging Face daily papers·Sep 2, 20:00 UTC · 13d agoAI research

55

Mask Forcing: Improving Autoregressive Video Diffusion Distillation via Dual-Noise Masking Rollout

Researchers propose Mask Forcing, a dual-noise masking rollout that mitigates mode collapse in autoregressive video diffusion distillation.

The paper targets over-saturation and over-smoothing in autoregressive video diffusion models distilled via Distribution Matching Distillation, attributed to reverse-KL mode-seeking behavior. Mask Forcing perturbs the student self-rollout with random masks along spatial and temporal axes, injecting cleaner tokens that act as denoising guidance for noisier tokens. Experiments show improved visual quality across multiple distillation methods without using real video data or extra post-training stages.

Hugging Face daily papers·Sep 7, 20:00 UTC · 8d agoAI research

22

RelightFormer: Feed-forward Generative Transformer for Multiview Object Relighting

RelightFormer is a feed-forward generative transformer for photorealistic single- and multi-view object relighting, trained on a 90K-object dataset.

Researchers introduce RelightFormer, a feed-forward generative transformer adapted from a video foundation model that performs direct image relighting without explicit intrinsic property estimation. The architecture injects target environment maps via a latent illumination module with cross-attention and uses permutation-invariant positional encodings for unordered multi-view inputs. Training relies on the newly constructed Laval Objaverse Dataset (LOD) with 90K objects and 39K unique illuminations, and the model shows state-of-the-art quality with strong zero-shot generalization across single-view, multi-view, and novel-view relighting.

Hugging Face daily papers·Sep 6, 20:00 UTC · 9d agoAI research

38

UniMate: One Unified Model to Animate Diverse Skeletons

Researchers introduce UniMate, a topology-aware diffusion transformer generating text-driven motion for arbitrary 3D skeletons without per-skeleton retraining.

UniMate is a unified foundation model that synthesizes articulated motion for arbitrary skeletons from a rigged 3D asset and a text prompt, with no test-time optimization or per-skeleton fine-tuning. It uses a topology-aware diffusion transformer combining graph-aware attention bias, a spectral rotary position embedding generalizing RoPE via the graph Laplacian, and a rest-pose topological conditioner. Trained on UniML3D, a curated set of 13,006 motion sequences spanning bipedal to serpentine skeletons, it outperforms state-of-the-art baselines and supports zero-shot cross-topology transfer, in-betweening, and text-guided editing.

arXiv cs.AI / cs.LG / cs.CL·Sep 4, 17:59 UTC · 11d agoAI research

16

FreeFlow: A Bias-free Hierarchical Transformer for Optical Flow Estimation

FreeFlow is a bias-free hierarchical transformer achieving state-of-the-art optical flow results on Sintel, KITTI-2015, and Spring benchmarks.

FreeFlow replaces task-specific inductive biases like correlation volumes and iterative warping with a single feed-forward encoder-decoder combining window, shifted-window, and reduced-resolution global attention. It reaches 0.68/1.48 EPE on Sintel Clean/Final, 3.23 Fl-all on KITTI-2015, and 3.192 1px on Spring. The architecture scales consistently from small to large variants and remains memory efficient at 1080p inference.

Hugging Face daily papers·Sep 9, 20:00 UTC · 6d agoAI research

30

A Princeton Researcher Proposes Recurrent Looped Transformer (RLT) that Carries Decoder State across Every Token, Fixing 96 Blocks per Token with Unbounded Temporal Depth

Princeton researcher Yifan Zhang proposes Recurrent Looped Transformer, carrying full decoder state across every token for unbounded temporal depth.

Yifan Zhang's technical report defines the Recurrent Looped Transformer (RLT), pairing a causal encoder with a recurrent decoder whose final output and layerwise sliding-window attention cache carry into every subsequent token with no prompt-response boundary reset. The reference configuration ties 48 encoder and 48 decoder layers, executing 96 logical blocks per token while the state path grows to 48t blocks after t tokens at fixed per-token compute. The report details RL replay contracts that rebuild all states under current parameters and exact prefix snapshots for multi-turn serving, but explicitly reports no measured efficiency, reasoning quality, or scaling results.

MarkTechPost·Sep 13, 17:02 UTC · 2d agoAI research1

40

Local gradient neural operator

Researchers propose LGNO, a lightweight interpretable neural operator using learnable local stencils, matching global-operator accuracy on PDE benchmarks with fewer parameters.

LGNO builds on nonlinear gradient discretization priors and uses multilayer perceptron convolutional layers to learn translation-invariant local kernels resembling discrete stencils. A zero consistent stencil factorization separates coefficient learning from field reconstruction, and network folding shares equivalent components to cut parameter counts for symmetric problems. Evaluations on linear and nonlinear, static and dynamic, and low- and high-dimensional PDE benchmarks show maintained accuracy, parameter efficiency, and rollout stability, with applicability to diffusion, flow, and quantum problems.

arXiv cs.AI / cs.LG / cs.CL·Sep 7, 16:53 UTC · 8d agoAI research

28

Distance generalization in transformers: why bother with positional encoding?

arXiv study uses synthetic delay-copy tasks to show how RoPE, ALiBi, NoPE and training data diversity affect transformers' distance generalization.

The paper studies distance generalization in transformers: extrapolating when inter-token distances change between training and inference while context length stays fixed. Using two synthetic delay-copy tasks with finite source-recall distances, the authors test models on unseen delays. They investigate whether positional encodings such as RoPE and ALiBi outperform no positional encoding (NoPE), how the diversity of training distances affects performance, and when distance transfer learning is positive or negative.

arXiv cs.AI / cs.LG / cs.CL·Sep 10, 17:57 UTC · 5d agoAI research2

30

Learning Sparse Decision Trees via Transformer Variational Auto-Encoders

TREVIS uses a Tree Transformer VAE latent space to learn decision trees matching near-optimal predictive performance while improving structural sparsity.

TREVIS learns decision trees optimized for complex objectives by exploring the latent space of a Tree Transformer Variational Auto-Encoder (TTVAE). Mapping trees to continuous latent representations replaces the discrete search space with a continuous one, enabling gradient-based optimization through a differentiable surrogate model. Experiments show TREVIS matches the predictive performance of near-optimal algorithms while improving structural sparsity, targeting high-stakes contexts needing transparent decision logic.

Hugging Face daily papers·Aug 31, 20:00 UTC · 15d agoAI research

18

UniMate: One Unified Model to Animate Diverse Skeletons

UniMate is a topology-aware diffusion transformer generating articulated motion for arbitrary rigged skeletons from text, trained on 13,006 motion sequences.

UniMate is a unified foundation model that animates arbitrary rigged 3D skeletons from an asset and text prompt with no test-time optimization or per-skeleton retraining. It uses a topology-aware diffusion transformer combining graph-aware attention bias from joint relations and geodesic distances, a spectral rotary position embedding generalizing RoPE to kinematic trees via the graph Laplacian, and a global topological conditioner. The accompanying UniML3D dataset spans 13,006 motion sequences across bipedal, quadrupedal, avian, marine, insectoid, serpentine, and articulated rigid-object skeletons; the model outperforms baselines and supports zero-shot cross-topology transfer, in-betweening, expansion, and text-guided editing.

Hugging Face daily papers·Sep 3, 20:00 UTC · 12d agoAI research

30

LynnReal-Omni: Native multi-modal Video Generation for Agentic Visual Workflows

LynnReal-Omni unifies controllable video generation tasks in a 32B multimodal diffusion transformer, with a 27B Flash variant rendering 540p clips in 377 ms.

LynnReal-Omni is a native multimodal video generation framework built on a 32B shared multimodal diffusion transformer unifying text-to-video, image-conditioned generation, reference guidance, structural control, editing, restoration and long-video generation, accepting heterogeneous inputs like 3D renders and game recordings for agentic visual workflows. A dedicated 27B Flash model enables real-time rendering, producing a 22-frame 540p video in 377 ms on one H100 versus 843 ms for the full model. The work introduces a curated multi-shot audiovisual data pipeline and MSAVP, a 100-prompt, 20-metric evaluation design covering instruction following, plausibility, visual quality, temporal behavior and audio coordination.

Hugging Face daily papers·Sep 13, 20:00 UTC · 2d agoAI research

28

Mi-Ripple: Restoring Images Degraded by Iterative AI Editing

Mi-Ripple is a diagnosis-guided restoration workflow that removes digital ripple artifacts introduced by iterative AI image editing while preserving structure.

Iterative reference-conditioned image editing can introduce grid-like and granular textures known as digital ripple. Mi-Ripple separates periodic lattice artifacts from content-entangled granular texture, then applies selective spectral notching, structure-aware smoothing, and cleaned-reference regeneration. In fourteen notch-only executions, whole-image residual standard deviation was 0.08-0.44 in CIELAB lightness units, and reference cleaning reduced output debris density by 45% in a paired example.

Hugging Face daily papers·Sep 9, 20:00 UTC · 6d agoAI research

25

Graph Machine: Towards Better Pretraining via Edges

Researchers propose Graph Machine, an O(n)-state sparse architecture that replaces 75% of Qwen3-0.6B dense layers with only slight loss change.

The paper introduces the Graph Machine (GM), an architecture that maintains an O(n)-sized state accessed through sparse, dynamic routing via pointer-like edges updated differentiably by a referral mechanism resembling pointer chasing. The authors replaced 75% of dense Transformer layers in Qwen3-0.6B with GM sparse layers and pretrained from scratch on 15.7B tokens. Retrieving 2 of 4,096 tokens per KV head in each sparse layer degrades loss only slightly, while retrieving 4 marginally improves loss over the dense baseline.

Hugging Face daily papers·Sep 1, 20:00 UTC · 14d agoAI research

30

LLaDA-UI: Bringing Block-wise Diffusion to Vision-Language GUI Agents

LLaDA-UI, a 16.7B block-wise diffusion vision-language GUI agent, outperforms Qwen2.5-VL-7B and beats Qwen3-VL-8B on four of six GUI benchmarks.

LLaDA-UI is a 16.7B-parameter MoE-based, block-wise diffusion vision-language GUI agent built on the LLaDA2.0-mini-base diffusion language backbone with a native-resolution vision encoder. It uses a two-stage pipeline: general multimodal pre-training followed by GUI-agent supervised fine-tuning on mobile, desktop, web, and grounding data. It substantially outperforms Qwen2.5-VL-7B and surpasses Qwen3-VL-8B on four of six reported GUI benchmarks, establishing block-wise diffusion as a practical paradigm for latency-sensitive multimodal agents.

Hugging Face daily papers·Sep 8, 20:00 UTC · 7d agoAI research

48

NOAH: Learning the Full Patient Journey. A Longitudinal Multimodal Time-Aware Model for Representation and Forecasting

Researchers introduce NOAH, a generative time-aware transformer trained on 559 million MIMIC clinical events to model and forecast patient trajectories.

NOAH is a task-agnostic, time-aware generative transformer designed to represent and forecast the full multimodal patient journey across medical images, time-series signals, categorical events, and clinical text. It was trained on over 559 million clinical events from 431,000 hospital visits covering 299,000 patients in the MIMIC dataset family. The architecture combines bidirectional time integration with a variational latent space to capture continuous patient state evolution and clinical stochasticity. NOAH supports autoregressive forecasting with time control, zero-shot classification, and counterfactual intervention simulation, with evaluations on 15 ICD chapters, 29 comorbidities, and time-to-event prediction.

arXiv cs.AI / cs.LG / cs.CL·Sep 8, 17:56 UTC · 7d agoAI research2

40

Attention-DP3: Spatially Object-aware 3D Diffusion Policy via Geometry-aligned Attentional Conditioning

Attention-DP3 adds spatially object-aware attentional conditioning to 3D diffusion policies, improving robotic manipulation by up to 31% under heavy clutter.

Attention-DP3 injects object-level geometric cues into the unchanged DP3 diffusion policy via Tri-field Attentional Conditioning, using targetness, intra-target saliency, and backgroundness fields. Open-vocabulary 2D segmentation masks are lifted to 3D with calibrated camera geometry to build object-centric priors. Experiments on Adroit, DexArt, MetaWorld, and a real-world SO101 platform show state-of-the-art results, outperforming DP3 by up to 31% under heavy distractor clutter; the code is publicly available on GitHub.

Hugging Face daily papers·Sep 9, 20:00 UTC · 6d agoAI research

15

NOAH: Learning the Full Patient Journey. A Longitudinal Multimodal Time-Aware Model for Representation and Forecasting

Researchers introduce NOAH, a time-aware generative transformer trained on 559 million MIMIC clinical events to forecast multimodal patient trajectories.

NOAH is a task-agnostic, time-aware generative transformer trained on over 559 million clinical events from 431,000 hospital visits by 299,000 patients across the MIMIC dataset family. It uses bidirectional time integration and a variational latent space to model the stochastic evolution of patient states, natively processing medical images, time-series signals, categorical events, and structured or unstructured clinical records. The model supports autoregressive forecasting with optional time control, zero-shot classification, and counterfactual intervention simulation, with strong probing performance across clinical outcomes, 15 ICD chapters, and 29 comorbidities.

Hugging Face daily papers·Sep 7, 20:00 UTC · 8d agoAI research1

42

CanvasAnneal: Curriculum Reinforcement Learning for Diffusion Language Models

CanvasAnneal injects teacher reasoning traces into diffusion canvases during curriculum RL, improving diffusion LLMs on MATH500, Countdown, and Tau2.

CanvasAnneal is a curriculum-guided reinforcement learning framework for diffusion language models that addresses exploration bottlenecks in standard RL. It warm-starts exploration by injecting teacher-generated reasoning traces into the initial diffusion canvas, then gradually removes this guidance so the model generates reasoning trajectories independently. Across mathematical reasoning and tool-use benchmarks, it improves over standard diffu-GRPO on MATH500, Countdown, and Tau2 and accelerates reward improvement, though gains are task-dependent.

arXiv cs.AI / cs.LG / cs.CL·Sep 11, 16:59 UTC · 4d agoAI research1

28

Anatomical Grounding and Leakage-Aware Multimodal Contrastive Learning for Alzheimer's Disease Classification from Structural MRI

Study of label leakage and anatomical grounding in multimodal MRI models for Alzheimer's staging shows cognitive-score fusion accuracy of 87.3% is leakage-driven.

The authors train a ResNet18 slice-based encoder with a one-layer Transformer on 1,075 ADNI-1 T1 MRI scans, using FastSurfer segmentations and YOLOv8 localization (mAP_50 above 0.96) as anatomical reference. Grad-CAM shows the image-only classifier often attends to skull and background rather than disease-relevant structures. A CLIP-style image-tabular contrastive framework organized along a label-leakage spectrum yields 87.3% three-way accuracy with cognitive scores versus 73.0% with regional volumes, and cropping to the medial temporal lobe raises image-only accuracy from 58.7% to 65.1%. Results come from single runs on a small balanced test set with reported confidence intervals.

arXiv cs.AI / cs.LG / cs.CL·Sep 14, 17:13 UTC · 1d agoAI research1

16

Bridging Control, Inference, Transport, and Thermodynamics: From Theory to Applications in Learning

Review connects control theory, optimal transport, probabilistic inference, thermodynamics, and machine learning via free-energy optimization under constraints.

The review unifies five fields: control theory, optimal transport, probabilistic inference, non-equilibrium thermodynamics, and machine learning. The common conceptual thread is optimization of free-energy-like functionals under dynamical or statistical constraints. Selected applications are presented in reinforcement learning, variational inference, and generative modeling. The tutorial-style text assumes no prior familiarity and begins from physics principles.

arXiv cs.AI / cs.LG / cs.CL·Sep 14, 17:16 UTC · 1d agoAI research

15