Skip to content
NewA calmer, faster ZeroHour

FrontierML tests whether AI agents can buy better model scores

Original title: Import AI 481: AI sub-contractors; a wet lab for AI; and subterfuge during development

Import AIby Jack Clark ()12 min read1
AI summary · x-ai/grok-4.7Summaries by Vani AI

Opus 5.5 gained 58.8 points with a $200 budget versus 66.1 for a human researcher baseline.

Import AI describes FrontierML, a benchmark of 21 long-horizon machine-learning tasks in which agents get a 12-hour CPU sandbox, limited validation samples, and $10, $50, or $200 to spend on compute and data. Stronger agents improved more as spending rose; Opus 5.5 led reported systems, and Kimi K3.1 Pro was the top open-weight model cited. The newsletter also notes Y Combinator startup Petrichor, a wet lab meant to take experiment requests from AI agents, and Scale AI’s RSI Control Arena for testing covert side tasks during self-development. The arena write-up is cut off before results.

Unconfirmed: The RSI Control Arena section ends before describing outcomes, so its findings are not established here.

Key facts
  • Tasks cover LLM training, efficiency, vision and audio, computer use, applied NLP, and math and games.
  • A human baseline rose 56 points at $10 and 66.1 points at $200.
  • Kimi K3.1 Pro scored 25.5 at $10 and 29.6 at $200.
  • In 28% of trajectories, agents trained new components or a model from scratch.
  • Petrichor says models can submit experiments in plain language and receive results without a person relaying them.
Explain this AI news with Vani AIWhat was announced, what is new and why it matters, in a new chat

From the original article

12 min read at importai.substack.com

Welcome to Import AI, a newsletter about AI research. Import AI runs on arXiv, cappuccinos, and feedback from readers. If you’d like to support this, please subscribe.

Import AI

By Jack Clark · importai.substack.com

Read the full article at Import AI

A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.