METR updates AI time-horizon chart to include open-ended tasks
Understand this story
The AI evaluation nonprofit METR updated its AI task time-horizon chart, adding longer-duration and more open-ended tasks. The new report finds that some AI capabilities are still improving along an exponential trajectory, but models perform worse on tasks requiring judgment than on programming tasks. Among tasks in which models were asked to design AI evaluations, none of the AI-designed evaluations reached a level that METR staff considered usable for research; on the task of scoring philosophical arguments, models also underperformed humans. The report says the latest models, such as Opus 5.5, have reached a "barely passing" level on such tasks, whereas a year ago all models were "clearly below standard." Completion times for the tasks in the new chart are mostly estimates, because paying people to complete tasks that might take hundreds of hours, or that only a few experts can complete, would be too costly. In the philosophical argument task, most arguments were scored by only one person, and models were judged by how closely they agreed with that scorer.
AI overview based on source reports · Updated 59 min ago
Coverage timeline
Follow the coverage to explore different perspectives.
- MIT Technology Review · AIThe most misunderstood graph in AI just got a major update
METR's updated time horizon plot adds tasks that take humans longer and are more open-ended, and finds that models still fall short of human performance on judgment-heavy work such as designing AI evaluations and scoring philosophical arguments. The report notes that no AI-designed evaluation was good enough for METR staff to consider useful for their research, though newer models like Opus 5.5 reach "borderline above the bar" performance where a year ago all models scored "clearly below the bar." METR cautions that hard-to-verify tasks are difficult to quantify, with completion times mostly estimated rather than measured.
Activity for this story
Not enough continuous observations to show a trend yet.