Inside NVIDIA’s cuDNN Graph API: Fusion, Autotuning, and Plan Reuse with cuDNN Frontend
MarkTechPost tutorial walks through NVIDIA's cuDNN Frontend graph API, covering kernel fusion, autotuning, plan reuse, and CUDA graph capture on Colab GPUs.
The tutorial explains how to express GPU computations as operation graphs via the cuDNN Frontend graph API, running the five-step build pipeline of validate, build operation graph, create execution plans, check support, and build plans. It progresses from a single fused convolution with bias and ReLU to autotuning across engine configs, FP8-style epilogues, attention, plan serialization, dynamic shapes, and CUDA graph capture. Each kernel is benchmarked against a PyTorch reference on a single Colab GPU to verify correctness and measure cost. The piece also covers practical setup issues like making libcudnn.so visible to the frontend's dynamic loader.
- Describes cuDNN Frontend's five-step execution plan build pipeline in detail
- Covers autotuning engine configs, FP8 epilogues, attention, and dynamic shapes
- All benchmarks verified against PyTorch references on a single Colab GPU
- Includes workarounds for resolving the frontend's dynamic loader against libcudnn
From the original article
11 min read at marktechpost.com
Learn how to leverage NVIDIA’s cuDNN Frontend Graph API to build custom kernel fusions, autotuning engine configurations, FP8-style epilogues, scaled dot-product attention, dynamic shapes, and CUDA graph captures.
By Sana Hassan · marktechpost.com
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.