iCreatAI News
Skip to content
Trending storyUpdating

METR warns AI capabilities are growing faster than safety evaluation capacity

1 reports1 sources47 min agoUpdated

Understand this story

AI overview

The AI research organization METR warns that existing public data is insufficient to rule out a sharp acceleration in AI research in the coming years, which could make it impossible to understand, evaluate, and respond to the catastrophic risks posed by new AI systems. METR says the pace at which AI model capabilities are improving is outpacing experts' ability to reliably assess their safety, and the gap between the two constitutes the core of the aforementioned risk. This judgment is based on public data; METR did not specify particular models or a timeline.

AI overview based on source reports · Updated 15 min ago

Coverage timeline

Follow the coverage to explore different perspectives.

Nov 22
  1. The Decoder
    AI is getting better faster than experts can reliably test whether new systems are safe, METR warns

    METR warns that public data is insufficient to rule out a drastic acceleration in AI research over the next few years, which could leave humans unable to understand, assess, and react to catastrophic risks from new AI systems. The organization found leading models already beat experienced human professionals most of the time on clearly defined coding and computational tasks with direct feedback, and the task length they can handle doubles every 3.6 months, with a naive extrapolation reaching a 50 percent success rate on 200-hour tasks by July 2027. METR also warns that models still struggle with open-ended questions lacking immediate feedback and that easy-to-run tests may overestimate real-world performance.

Activity for this story

Not enough continuous observations to show a trend yet.