METR warns AI gains are outpacing reliable safety tests
Original title: AI is getting better faster than experts can reliably test whether new systems are safe, METR warns
On defined coding tasks, models already beat experienced professionals most of the time, and handleable task length doubles every 3.6 months.
METR says current public data cannot rule out a sharp acceleration in AI research over the next few years, which could leave people unable to assess catastrophic risks from new systems. On clearly defined coding and computational tasks with direct feedback, models beat experienced professionals most of the time but have not outperformed large groups of top experts. A naive extrapolation of METR's data puts a 50 percent success rate on 200-hour tasks by July 2027. The group also says easy tests may overestimate real-world performance.
Unconfirmed: The July 2027 date is METR's naive extrapolation, not a confirmed forecast.
- Handleable task length doubles every 3.6 months, according to METR.
- A naive extrapolation reaches 50 percent success on 200-hour tasks by July 2027.
- The capability chart runs through Opus 5.5 and GPT-6 Astra.
- Models still struggle with open-ended questions that lack immediate feedback.
- Organizations
- METR
- AI models
- Opus 5.5GPT-6 Astra
From the original article
1 min read at the-decoder.comMETR reports that leading AI models like Opus 5. 5 and GPT-6 Astra are improving rapidly at research tasks but getting harder to measure. Current data can't rule out a dangerous acceleration in AI development, the organization warns, though it also says common tests may overestimate what the models can actually do.
By Matthias Bastian · the-decoder.com
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.