DuoDiscover searches optimizations with cheap proxy evaluators
Original title: DuoDiscover: Scaling Autonomous Discovery Beyond the Evaluation Bottleneck
Authors say it doubles expert speedup on SWE-fficiency and reports 1.92 times vLLM and 1.38 times SQLite.
DuoDiscover is a search system in which coding agents jointly invent an optimization target and a fast proxy evaluator, called a yardstick, then hill-climb that proxy. Fidelity tests recheck rankings against true evaluations and invalidate drifting yardsticks, while a lightweight orchestrator manages a bottom-up agent fleet. The authors demonstrate it on two repository benchmarks and four GPU and TPU kernel suites. They also report beating AlphaEvolve-based ArchAgent v2 and the best human design on a CPU microarchitecture task whose evaluations can take up to 28 hours.
Unconfirmed: Speedups and rankings are claims from the paper and are not independently confirmed here.
- On SOL-Exec it ranks first on 121 of 235 kernels and top three on 212.
- vLLM and SQLite gains are 1.92 times and 1.38 times over base code.
- The CPU task took 16.7 days versus more than 40 days for the compared run.
- Reported microarchitecture gain is 1.65 times ArchAgent v2's gain.
From the original article
1 min read at huggingface.co
Evaluation has been central to advances in AI, and it is equally critical to autonomous discovery. Yet scaling autonomous discovery to real-world engineering challenges faces a fundamental bottleneck: the cost of evaluating progress. In real settings, a single true evaluation, such as a full benchmark or serving trace, can take hours, so a search can afford only a handful of true evaluations.
By Shubham Agarwal, Akshat Gupta, Mike Heddes · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.