ReproBridge lifts ML reproduction success from 76.4% to 92.6%
Original title: ReproBridge: Building Reproducibility into ML Research Artifacts
Across 1,029 tasks from 387 experiments, refined artifacts also cut mean API spending by 19.7%.
ReproBridge is an agent workflow that packages recovered procedures into a refined codebase, environment setup, launchers, and a reproduction guide. On ReproBridge-Bench, four models reproduced experiments more often from those artifacts than from raw code, and the cross-model success spread narrowed from 21.6 to 3.9 percentage points. Ablations found that removing any component reduced success, with code and environment removal causing the largest losses.
Unconfirmed: Success and cost figures are the authors' measurements; the four models are not named.
- The benchmark spans 36 papers and five human-verified reference scores per task.
- All four evaluated models exceeded 90% mean success with refined artifacts.
- A separate Auditor checks execution evidence from fresh agent sessions.
- Mean API spending fell 19.7% when agents used the refined artifacts.
From the original article
1 min read at huggingface.co
Reproducing machine learning experiments requires reconstructing procedures from information scattered across papers, released code, and supporting artifacts. For coding agents, reproduction success can therefore depend on their ability to fill in missing details, rather than on the released artifacts alone. We ask whether preserving recovered procedures and decisions improves subsequent independent reproduction.
By Yubo Wang, Rui Meng, Jaehyun Nam · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.