ReMoMask-2: Latent Retrieval-Augmented Masked Motion Generation
ReMoMask-2 rebuilds retrieval in the generator's latent space for text-to-motion generation, achieving lowest FID on KIT-ML and SnapMoGen.
ReMoMask-2 is a retrieval-augmented text-to-motion framework that constructs its retrieval database directly in the generator's pre-quantization latent space and aligns text queries through a distilled lightweight projector, eliminating the representation gap. The framework combines Hierarchical Bidirectional Momentum contrastive learning, Semantic Spatial-Temporal Attention, and Topology Structured Masking to handle hierarchical motion structure. The retriever achieves state-of-the-art accuracy, and ReMoMask-2 attains the lowest FID on KIT-ML and SnapMoGen, with a single mask-transformer stage outperforming the previous two-stage pipeline while delivering the fastest inference.
- Latent-space retrieval database closes representation gap in RAG-T2M
- Retriever achieves state-of-the-art accuracy on HumanML3D, KIT-ML, SnapMoGen
- Lowest FID on KIT-ML and SnapMoGen benchmarks
- Single mask-transformer stage beats prior two-stage pipeline at fastest inference
From the original article
1 min read at huggingface.co
Text-to-motion (T2M) generation maps natural language to human joint movements, aiding gaming, VR, and robotics. Retrieval-Augmented Text-to-Motion (RAG-T2M) improves generation on complex descriptions by conditioning on retrieved motion-text pairs.
By Yiran Wang, Zeyu Zhang, Ling Shao · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.