TIDE speeds FSDP2 with zero-copy layout-aware collectives
On 64 GPUs, step time falls by up to 27.1% versus stock FSDP2 and 6.0% versus FSDP2+SymmMem on three training workloads.
1 stories, newest first · one card per story
On 64 GPUs, step time falls by up to 27.1% versus stock FSDP2 and 6.0% versus FSDP2+SymmMem on three training workloads.
Today 349 readers · 375 views · 292 reads · all time 158K views