Columbia and CertiK introduce TensorLean to verify GPU kernels
Original title: A Team of Researchers from Columbia and CertiK Introduce TensorLean for Formal Verification of GPU Kernels
Of 149 kernel instances checked, 83 were proved correct and the rest led to 38 reported bugs.
Researchers from Columbia University and CertiK introduced TensorLean, a Lean-based verifier that checks compiled GPU kernels at the PTX level against a tensor-level math specification. It covers data races, termination, output within tolerance and untouched memory, using Lean, Z3 and cvc5. The team checked kernels including FlashAttention-2, -3 and -4 and DeepGEMM. Specs totaled 10,278 lines against 151,285 lines of kernel source.
- It checks PTX compiled for A100, H100 and B200 kernels.
- FlashAttention accounts for 23 of the 38 bugs reported upstream.
- Bugs include 19 memory access errors, 13 numerical bugs and 7 data races.
- A FlashAttention-4 data race never appeared in 17,760 test runs.
- Median check time is 10 minutes, with a 50-minute maximum.
- Products
- TensorLeanFlashAttention
- Organizations
- Columbia UniversityCertiK
From the original article
8 min read at marktechpost.comTensorLean, from Columbia University and CertiK, formally verifies production GPU kernels at the PTX level using Lean. It checked 149 kernel instances, proved 83 correct, and found 38 bugs in FlashAttention, DeepGEMM and more. The post A Team of Researchers from Columbia and CertiK Introduce TensorLean for Formal Verification of GPU Kernels appeared first on MarkTechPost.
By Michal Sutter · marktechpost.com
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.