MetaKernelBench: Measuring GPU Kernel Knowledge Transfer Beyond Code
MetaKernelBench measures whether GPU kernel skills transfer across CuTe and TIRx on 74 problems.
MetaKernelBench measures whether experience distilled from a GPU kernel attempt in one domain-specific language improves a fresh attempt at the same problem in another. Its 74 problems are fused subgraphs in six families, each posed as paired CuTe DSL and TIRx variants that differ only in the DSL. Across six models and both directions, paired lift over matched solo attempts ranged from -19% to +29%, while regressions occurred on 16% to 45% of problems. Outcomes tracked whether the source beat the target's solo result: improvement in 71% of comparisons when it stood above, and regression in 54% when it stood below.
- Seventy-four fused-subgraph problems are posed in both CuTe DSL and TIRx.
- Natural-language skills transfer even if the source attempt fails verification.
- Paired lift over solo attempts ranged from -19% to +29% across six models.
- Regressions occurred on 16% to 45% of problems in every setting.
From the original article
1 min read at huggingface.co
Recent GPU kernel optimization agents retain what they learn in knowledge bases or as distilled skills. Kernel benchmarks score each attempt's implementation for correctness and speed but leave the reuse value of retained experience unmeasured.
By Xueyi Chen, Shiyu Liu, Xin Jin · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.