SkyPilot says rack-local scheduling yields 27% more usable GPUs
Original title: Grace Blackwell needs a new scheduler: 27% more usable GPUs from the same racks
Topology-aware placement, in-place recovery, and defragmentation keep multi-node jobs on NVL72 NVLink instead of InfiniBand.
SkyPilot describes how NVIDIA NVL72 puts all 72 GPUs in a rack on one NVLink fabric, while jobs that cross racks still fall back to InfiniBand. Tensor and expert parallelism sit on that path: a cited Hugging Face measurement found a 43% training-throughput drop off NVLink, and unoptimized all-to-all can take up to 60% of MoE training time. The post says SkyPilot preserves rack locality with topology-aware placement, in-place hardware-failure recovery, and automatic cluster defragmentation.
Unconfirmed: The 27% usable-GPU gain is SkyPilot's own claim, not an independent measurement.
- Hugging Face measured a 43% training throughput drop moving tensor parallelism from NVLink to InfiniBand.
- On 8-GPU servers, unoptimized 64-way expert-parallel all-to-all can use up to 60% of training time.
- A vLLM benchmark reported 13.9x more KV-cache blocks and 3.9x throughput across two H100s.
- NVL72 connects all 72 GPUs in a rack with a unified NVLink fabric.
From the original article
10 min read at skypilot.ai
Not all high-speed interconnects are created equal, and using the wrong one can be quite costly: running the same distributed training job over InfiniBand rather than NVLink can degrade performance by 43% . Historically, NVLink was limited to communication within a single node, meaning that multi-node AI workloads were forced to communicate over InfiniBand or a cloud equivalent (EFA on AWS, GPUDirect-TCPX on GCP).
By wondersky · skypilot.ai
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.