Skip to content
NewA calmer, faster ZeroHour
StoryAI researchinfo

ReproBridge artifacts lift ML reproduction success to 92.6%

2 sources · 2 articlesFirst reported Updated ()
What's new

First merged summary.

Merged summary · x-ai/grok-4.7 · rewritten as coverage arrivesSummaries by Vani AI

Refined ReproBridge artifacts raise four models' mean reproduction success from 76.4% to 92.6% and cut API spending 19.7%.

ReproBridge is an agent workflow that turns scattered paper, code, and artifact procedures into a refined codebase, environment setup, launchers, and a reproduction guide. On ReproBridge-Bench—1,029 tasks from 387 experiments across 36 papers, with five human-verified reference scores per task—four models reproduced experiments more often from those artifacts than from raw code. Mean success rose from 76.4% to 92.6%, every model exceeded 90%, and the cross-model success spread narrowed from 21.6 to 3.9 percentage points. Mean API spending fell 19.7%. Ablations found that removing any component reduced success, with code and environment removal causing the largest losses. A separate Auditor checks execution evidence from fresh agent sessions.

Key facts
  • Benchmark covers 1,029 tasks from 387 experiments across 36 papers.
  • Mean success rose from 76.4% to 92.6% with refined artifacts.
  • All four evaluated models exceeded 90% mean success.
  • Cross-model success spread narrowed from 21.6 to 3.9 percentage points.
  • Mean API spending fell 19.7% when agents used refined artifacts.
  • Removing code or environment components caused the largest success losses.
Explain this AI news with Vani AIWhat was announced, what is new and why it matters, in a new chat

What each outlet adds

beyond the shared facts
  • Hugging Face daily papersSuccess-spread narrowing, ablations, Auditor, and five reference scores per task.
  • arXiv cs.AI / cs.LG / cs.CLStates the 76.4% to 92.6% mean success rise.
Products
ReproBridgeReproBridge-Bench

Coverage

oldest first · one row per article · UTC
  1. · 6d ago Hugging Face daily papersFirst report

    ReproBridge lifts ML reproduction success from 76.4% to 92.6%

    Across 1,029 tasks from 387 experiments, refined artifacts also cut mean API spending by 19.7%.

  2. · 6d ago arXiv cs.AI / cs.LG / cs.CL

    ReproBridge improves ML experiment reproducibility using refined artifacts

    Refined artifacts raise reproduction success from 76.4% to 92.6% and cut API spending by 19.7% on 1029 tasks from 36 papers.