METR: AI R&D capability doubles about every 3.6 months
Understand this story
METR released an evaluation of AI R&D-related capabilities up to Opus 5.5 and GPT-6 Astra, saying the relevant capabilities are improving rapidly and becoming harder to measure. On the combined TH1.1 and TH2 task set, the capability corresponds to a time horizon that doubles about every 3.6 months; looking only at the verifiable subset of TH2, it doubles about every 1.65 months.
AI overview based on source reports · Updated 53 min ago
Coverage timeline
Follow the coverage to explore different perspectives.
- METR BlogFeaturedAI R&D-relevant model capabilities are improving rapidly and are getting harder to measure
METR reports that public frontier models up to Opus 5.5 and GPT-6 Astra usually match or outperform recruited skilled humans on its coding and optimization tasks, with time horizons on the combined TH1.1 and TH2 suite doubling roughly every 3.6 months, though models show weaker judgment than human experts on tasks lacking clear empirical feedback. The post also details methodology changes, including fewer human baselines (directly used for only 9% of time estimates), multiple score thresholds per task, and a new IRT-based METR Capability Index, while cautioning that containerized evals may overestimate real-world capabilities because they resemble RL training environments.
Activity for this story
Not enough continuous observations to show a trend yet.