Skeleton Coding uses program structure to steer coding agents
Original title: Skeleton Coding: Program Structure as a Specification for Agents
Across four models and three benchmarks, the best skeletons cut mean test error 48.4% versus the best behavioral specs.
Researchers argue end-to-end tests cannot be known complete because the agent chooses program structure first. Skeleton Coding lets developers specify that structure in a code-like form together with behavioral tests. On four models and three benchmarks, the best skeletons cut mean test error 48.4% versus the best end-to-end specifications, and still help when a reference implementation can be queried.
- Skeletons narrowed the small-versus-large model gap by about 52.4%.
- Implementation cost fell by 15–26x.
- Code erosion dropped 24.7% on average.
- With a queryable reference, skeletons cut mean test error by up to 27.0%.
From the original article
1 min read at arxiv.orgCoding agents are typically provided end-to-end behavioral specifications that the code they produce should satisfy, such as tests and input--output examples. How many such specifications must a developer provide, and which ones, for the agent to produce the developer's intended program?
By Aditya Golatkar · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.