The Attention Triangle in Audio-Video Models
Researchers analyze the 'attention triangle' in audio-video diffusion models, showing bias-driven cross-attention routing causes semantic leakage and proposing inference-time interventions that improve grounding.
A study probes the three cross-attention edges linking text, audio, and video streams in audio-video diffusion models. It finds the audio-video edge is bidirectional and shaped by parameter-encoded biases, so prompts in tension with learned priors can be overridden, producing visually canonical but incorrect outputs. Attention-derived signals are used as diagnostics and to guide inference-time interventions that improve cross-modal semantic grounding while preserving generation quality.
24