KernelSeek beats FlashAttention-3 by asking questions, not adding tools
Original title: Ask, Don't Equip: Question-Driven Coding Agents for Deep Kernel Optimization
A tool-rich agent stalled at 0.50x after 133 steps; KernelSeek reached 1.13x, then 1.16x and 1.46x on later kernels.
KernelSeek is a question-driven coding agent for making an existing CUDA kernel faster without operator-specific knowledge or a faster implementation to copy. It pairs a general-purpose coding agent with a minimal harness and questions at decision points, first from a human and then from a bank the agent applies to itself. On the same task, evaluator, hardware, and week, a tool-rich autonomous agent plateaued at 0.50x FlashAttention-3 after 133 model-driven steps, while KernelSeek reached 1.13x. It later produced dependency-free FlashAttention forward kernels for Hopper and Blackwell, correct on all 182 scenarios per architecture.
Unconfirmed: The claim that every large gain followed a redirecting question, never added information, is the authors’ analysis of their traces.
- Hopper and Blackwell kernels are single CUDA files with no CUTLASS, CuTe, cuBLAS, or cuDNN.
- Geometric-mean speedups are 1.16x FlashAttention-3 and 1.46x FlashAttention-4.
- The run used 162 human messages and 11.5K agent tool calls.
- Authors say large gains followed questions that changed system layer or rejected a limit.
- Kernels, harness, question bank, and traces are released.
- Vendors
- NVIDIA
- AI models
- KernelSeek
From the original article
1 min read at huggingface.co
LLM agents for GPU kernel optimization sit at two ends of a structure-freedom tradeoff. Scripted agents fix what the model does at each step, via a learned policy, an optimization menu, or fixed roles, and are sample-efficient where the recipe applies and stall where it does not.
By Gabriele Oliaro, Jaeseong Lee, Zhaoyuan Su · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.