TIDE speeds FSDP2 with zero-copy layout-aware collectives
On 64 GPUs, step time falls by up to 27.1% versus stock FSDP2 and 6.0% versus FSDP2+SymmMem on three training workloads.
46
On 64 GPUs, step time falls by up to 27.1% versus stock FSDP2 and 6.0% versus FSDP2+SymmMem on three training workloads.
Today 840 readers · 1.1K views · 843 reads · all time 159K views