HRoPE encodes patch surfaces as slanted planes for multi-view attention
Plane parameters need no ground-truth geometry and can be learned from view synthesis alone or taken from foundation models.
Stories per day · 30 days
2 RSS
Relatedaction-control1computer-vision1game-ai1midi1multi-view1music-transcription1novel-view-synthesis1piano1transformer1transformers1video-generation1world-models1
7 stories, newest first · one card per story
Stories tagged #positional-encoding
ActionRoPE encodes game actions as world-coordinate positions
Coordinate-conditioned noise scheduling cut revisit drift 66% versus the strongest of four baselines.
DuET jointly transcribes piano notes and tracks beats for scores
On Gangqinpu, DuET cuts MUSTER mean error from 6.56% to 3.64% versus the strongest baseline.
How Local Mixing Encodes Relative Position in Global NoPE Attention
Hybrid local-mixing layers give global NoPE attention a recency bias that encodes relative position.
MinkowskiPE: Minkowski Positional Encoding for Spatiotemporal Perception
MinkowskiPE encodes relative spacetime displacement in attention and improves molecular and video prediction.
Paragraph Boundaries Are Not White Space:Compression Depth as the Signature of Hierarchical Structure
Compression depth, not location, distinguishes real paragraph structure under hierarchical rotary positional encoding.
Distance generalization in transformers: why bother with positional encoding?
arXiv study uses synthetic delay-copy tasks to show how RoPE, ALiBi, NoPE and training data diversity affect transformers' distance generalization.