AI is getting better faster than experts can reliably test whether new systems are safe, METR warns
AI is getting better faster than experts can reliably test whether new systems are safe, METR warns
METR warns that public data is insufficient to rule out a drastic acceleration in AI research over the next few years, which could leave humans unable to understand, assess, and react to catastrophic risks from new AI systems. The organization found leading models already beat experienced human professionals most of the time on clearly defined coding and computational tasks with direct feedback, and the task length they can handle doubles every 3.6 months, with a naive extrapolation reaching a 50 percent success rate on 200-hour tasks by July 2027. METR also warns that models still struggle with open-ended questions lacking immediate feedback and that easy-to-run tests may overestimate real-world performance.
Source: The Decoder · the-decoder.com