MLE-World predicts ML experiment outputs without running training
Original title: MLE-World: Predicting Machine Learning Experiment Outcomes with a Learned World Model
Finetuned from Qwen3.5-4B, it matched held-out terminal output on 77.9% of programs versus 41.1% for larger prompted models.
Researchers present MLE-World, a world model finetuned from Qwen3.5-4B that predicts a candidate program's error traces, training logs, and validation scores from code and a data description, plus a confidence score. Five test-time policies use those predictions, and a program is executed only when confidence is low. On 17 MLE-bench Lite tasks, the authors report up to 22.1% higher observed mean performance than an agent without the world model, with fewer real runs.
Unconfirmed: Performance and match rates are the authors' reported figures and are not independently verified.
- A prediction is one inference call rather than a full training run.
- Policies include Sequential revision, Best-of-N, Pairwise judging, Fusion, and a UCB-style rule.
- Held-out terminal-output match is reported at 77.9%, versus 41.1% for larger prompted models.
- The confidence score also supplies an exploration bonus.
- Products
- MLE-bench Lite
- AI models
- MLE-WorldQwen3.5-4B
From the original article
1 min read at huggingface.co
AI research agents can propose, execute, and improve programs that solve machine learning tasks. However, executing each candidate program requires training and evaluating ML models, limiting how many candidates an agent can explore at each step. We introduce MLE-World, a world model finetuned from Qwen3.
By Brian Ko, Han Wang, Hejia Geng · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.