TIDE speeds FSDP2 with zero-copy layout-aware collectives
On 64 GPUs, step time falls by up to 27.1% versus stock FSDP2 and 6.0% versus FSDP2+SymmMem on three training workloads.
46
- Mentions · 7d1
- Mentions · 30d1
- All mentions2
- First seenSep 12, 2026
- Last mentioned2d ago
Timeline
2 articles · newest firstFlattening Every Memory Peak in Long-Context Mixture-of-Experts Training
Four scheduling techniques bound all memory peaks in long-context MoE training, enabling 120B-667B models at 1M-token context with up to 10.4x throughput.
26
Appears with
- Wan2.2ai model · 1
- Llama-3.1 70Bai model · 1
- NVIDIAvendor · 1
- NCCLproduct · 1
- Qwen3-30B-A3Bai model · 1
Entities are extracted by the model from each article. First seen , last mentioned . Watching keeps it in this browser only (no account); the watchlist page and dashboard alerts use it.