FlashMask-3: Efficient and Expressive Mask-Aware Distributed Attention
FlashMask-3 adds Hopper and distributed mask-aware attention, raising long-context training throughput over prior kernels.
Stories per day · 30 days
2 stories, newest first · one card per story
Stories tagged #context-parallelism
Online Draft Co-Training for Speculative Decoding in Large-Scale, Long-Context RL Post-Training
NVIDIA researchers detail an end-to-end system for online draft co-training that speeds speculative decoding in large-scale long-context RL post-training.