Evaluating Coding Agents on Kernel Exploit Generation
KEX-bench finds coding agents rarely turn kernel bugs into exploit primitives without a reference PoC.
KEX-bench evaluates coding agents on generating exploit primitives against real operating-system kernels, with 45 tasks covering 40 Linux and Windows CVEs. Each task runs in an isolated virtual machine and uses a deterministic verifier for leaks, instruction-pointer control, heap read and write, and arbitrary address write. Without a reference proof of concept, the strongest configuration solved 1 of 20 Windows tasks and 14 of 25 Linux tasks; with a reference PoC it solved 31 of 45. The authors report that agents often reach kernel crashes but fail to shape kernel state into exploit primitives.
- Forty-five isolated-VM tasks cover 40 Linux and Windows kernel CVEs.
- Primitives include leaks, instruction-pointer control, heap access, and arbitrary writes.
- Without a reference PoC, best scores were 5% Windows and 56% Linux.
- Supplying a reference PoC raised the best result to 31 of 45 tasks.
From the original article
1 min read at arxiv.orgCoding agents now find real vulnerabilities in production software. However, bug discovery results do not measure whether agents can construct exploit primitives. We introduce KEX-bench, a benchmark for evaluating coding agents on exploit primitive generation against real operating-system kernels.
By Junyoung Jang · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.