MinkowskiPE: Minkowski Positional Encoding for Spatiotemporal Perception
MinkowskiPE encodes relative spacetime displacement in attention and improves molecular and video prediction.
Minkowski Positional Encoding parameterizes Lorentz transformations of query and key features from joint temporal and spatial coordinates. Attention scores depend on position only through relative spacetime displacement, so they are invariant to global coordinate translation, while remaining compatible with standard dot-product attention. It is evaluated on microscopic molecular dynamics and macroscopic video prediction. The method ranks best on all nine multi-trajectory molecular evaluations and reduces KTH video-prediction MSE by 9.9% versus the best baseline with roughly one-tenth as many parameters.
- Positional encoding via Lorentz transforms of query and key features.
- Scores depend only on relative spacetime displacement between tokens.
- Best on all nine multi-trajectory molecular dynamics evaluations.
- Cuts KTH video-prediction MSE 9.9% with about one-tenth the parameters.
From the original article
1 min read at huggingface.co
Modeling spatiotemporal coupling is a key challenge in building physical intelligence across scales, from microscopic to macroscopic. Existing models capture such structure broadly through physics-motivated dynamical formulations or learning-motivated architectures.
By Yuhao Li, Louie Hong Yao, Tianyi Shi · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.