EdgeKernelBench finds LLM edge kernels often correct but rarely fast
Original title: EdgeKernelBench: Unified LLM-Driven Kernel Generation for Edge Backends
The best single attempt scored 64.5% Correct@1 and 25.5% Fast@1.0; LEKA reached 43.5% Fast@1.0.
EdgeKernelBench pairs a unified IR deployment framework with a hierarchical benchmark spanning operators to compact model graphs on edge workloads. The IR records computation, scheduling, and memory decisions and reaches each backend through a compiler interface, instantiated with TVM. The Layered Edge Kernel Agent organizes optimization through a layered IR. On the agent subset it beat direct generation at 13.0% and Claude Code at 26.1% under the same base model and optimization budget.
- Edge hardware is heterogeneous, stacks fragmented, and resource budgets tight.
- Functional correctness did not consistently become performance gains.
- LEKA and Claude Code used the same base model and budget.
- The design was validated across heterogeneous edge backends.
- Products
- EdgeKernelBenchTVMClaude Code
- AI models
- LEKA
From the original article
1 min read at huggingface.co
Recent works show that LLMs can generate kernels with strong performance on server GPUs. Edge deployment presents distinct challenges because the hardware is heterogeneous, the software stacks are fragmented, and the resource budgets are tight. LLM kernel generation across diverse edge backends has yet to be studied systematically.
By Yuhua Zheng, Xueyi Chen, Xinze Zou · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.