AdaPRA adds adaptive process rubrics for data science agents
Original title: AdaPRA: Process Reward Agents with Adaptive Rubrics for Data Science
Five attempts per task raised an 80B Qwen model’s leaderboard percentile by 24.5 on DSPredict-Easy and 14.8 on DSPredict-Hard.
AdaPRA is a persistent coding-agent harness that keeps knowledge across attempts on the same data-science task and uses rubric-guided process supervision. A process reward agent audits the trajectory and workspace by running checks, and adapts rubrics per task. Authors say rubrics seeded on easier tasks transfer to harder ones, and that rubrics mined from one model’s runs improved DeepSeek-V3.2. They also report AdaPRA beating DataPRM and release code.
Unconfirmed: Leaderboard gains and the DataPRM comparison are the authors’ reported results.
- The harness accumulates knowledge across attempts on the same task.
- Rubrics seeded from easier tasks are said to generalize to harder tasks.
- Supervision mined from one base model’s runs improved DeepSeek-V3.2.
- Authors say AdaPRA outperforms DataPRM, a process reward model for data science.
- Code is shared for reproducibility.
- Products
- DataPRM
- AI models
- Qwen 80BDeepSeek-V3.2
From the original article
1 min read at huggingface.co
Data science agents can automate end-to-end machine learning workflows, yet they remain far from competitive on human leaderboards. Agents make process errors such as insufficient exploration, unsound validation, coding bugs, and poor use of a fixed budget (re-running the same exploratory data analysis and baseline models in every attempt).
By Atharva Naik, Vivek Chari, Pol Llado · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.