MIT Technology Review · AI· Grace Huckins·· 2 hr agoAI score60
The most misunderstood graph in AI just got a major update
The most misunderstood graph in AI just got a major update
AI summary
METR's updated time horizon plot adds longer and more open-ended tasks, and finds that while AI capabilities keep improving exponentially on verifiable work like coding, models still fall short of humans on judgment-heavy tasks—none of the AI-designed evaluations were deemed useful by METR staff, and models also underperformed humans at scoring philosophical arguments. The report notes that newer models like Opus 5.5 now reach "borderline above the bar" on evaluation design, up from "clearly below the bar" a year ago, but that hard-to-verify tasks remain difficult to quantify and extrapolate.
Source: MIT Technology Review · AI · technologyreview.com