EvoWeb benchmark tests coding agents on evolving web apps
Original title: EvoWeb: A Benchmark for Coding Agents on Evolving Web Applications
It uses 320 cumulative requests across 32 applications and a live-browser tester for integration and regressions.
EvoWeb evaluates coding agents that iteratively change a persistent codebase rather than building to a fixed specification. The benchmark has 320 cumulative requests over 32 web applications, grounded in real product features and organized by functional dependency graphs. A multimodal web agent checks each intermediate app in a live browser for new features, cross-request integration, and previously requested behavior. The authors say later, more dependent requests are solved less accurately and runtime errors increase over successive requests.
Unconfirmed: Performance gaps are as reported by the paper; model names and independent replication are not stated.
- Trajectory scores are aggregated across user-tolerance thresholds instead of one stopping point.
- Experiments cover open- and closed-source frontier models, which are not named.
- Cross-request integration is described as a key challenge.
- Deeper or more numerous dependencies correlate with lower accuracy.
From the original article
1 min read at huggingface.co
Web application development often proceeds through successive requests that extend an existing implementation. Yet web-generation benchmarks primarily evaluate development toward fixed specifications. We introduce EvoWeb, a novel benchmark comprising 320 cumulative requests over 32 web applications, grounded in real-world product features and structured by functional dependency graphs.
By David Nukrai, Bar Alon, Nativ Levy · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.