TIDE speeds FSDP2 with zero-copy layout-aware collectives
On 64 GPUs, step time falls by up to 27.1% versus stock FSDP2 and 6.0% versus FSDP2+SymmMem on three training workloads.
Stories per day · 30 days
1 RSS
4 stories, newest first · one card per story
Stories tagged #fsdp
Training Witnesses: Trusting the Training without Trusting the Trainer
Witnesses lets trainers certify language-model runs so others can audit training without full reproduction.
SlideDP: Scaling Host-Resident LLM Fine-Tuning Across Multiple GPUs
SlideDP scales host-resident multi-GPU LLM fine-tuning, beating offload baselines and nearing FSDP2 on Qwen3-14B.
Accelerating Sharded Data Parallelism at Scale with Federated Learning
Hybrid FL+FSDP and FL+HSDP algorithms cut communication overhead in sharded data parallelism, accelerating Llama3.1 8B pre-training on 512 A100 GPUs by up to 8x.