Retrofitting Code Using LLMs to Support Exceptional Behavior
EXCODER combines static/dynamic analysis with LLMs to retrofit exception-handling code, achieving 85.92% pass@1 with Qwen 2.5 Coder 32B on Java benchmarks.
The paper introduces the task of retrofitting existing code with Exception Related Code (throw statements, guarding conditions, try/catch blocks) so that given Exceptional Behavior Tests pass. EXCODER performs context engineering by integrating static and dynamic program analysis output with LLMs; it was evaluated on a benchmark built from 304 methods across 75 GitHub Java projects. Combined with Qwen 2.5 Coder 32B, EXCODER achieves pass@1, 5, and 10 rates of 85.92%, 86.18%, and 86.51%, roughly 13 percentage points over baseline, and manual inspection reveals remaining limitations.
- First test-driven approach to automatically generating missing exception-related code in existing codebases.
- Benchmark systematically removes ERC from 304 methods across 75 GitHub Java repositories.
- Pass@1 reaches 85.92% with Qwen 2.5 Coder 32B, 12.56 percentage points above baseline.
- Manual inspection of generated code identifies limitations and directions for future work.
- Products
- EXCODER
- AI models
- Qwen 2.5 Coder 32B
From the original article
1 min read at arxiv.orgException Related Code (ERC), which includes throw statements, conditions (if statements) that guard those throw statements, and try/catch blocks, is an essential component of software systems, allowing developers to detect and handle exceptional states that deviate from the expected program behavior. However, manually writing ERC across large codebases is tedious.
By Linghan Zhong · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.