iCreatAI News
Skip to content
Trending storyUpdating

METR: AI R&D capability doubles about every 3.6 months

1 reports1 sources13 hr agoUpdated

Understand this story

AI overview

METR released an evaluation of AI R&D-related capabilities up to Opus 5.5 and GPT-6 Astra, saying the relevant capabilities are improving rapidly and becoming harder to measure. On the combined TH1.1 and TH2 task set, the capability corresponds to a time horizon that doubles about every 3.6 months; looking only at the verifiable subset of TH2, it doubles about every 1.65 months.

AI overview based on source reports · Updated 53 min ago

Coverage timeline

Follow the coverage to explore different perspectives.

Nov 20
  1. METR BlogFeatured
    AI R&D-relevant model capabilities are improving rapidly and are getting harder to measure

    METR reports that public frontier models up to Opus 5.5 and GPT-6 Astra usually match or outperform recruited skilled humans on its coding and optimization tasks, with time horizons on the combined TH1.1 and TH2 suite doubling roughly every 3.6 months, though models show weaker judgment than human experts on tasks lacking clear empirical feedback. The post also details methodology changes, including fewer human baselines (directly used for only 9% of time estimates), multiple score thresholds per task, and a new IRT-based METR Capability Index, while cautioning that containerized evals may overestimate real-world capabilities because they resemble RL training environments.

Activity for this story

Not enough continuous observations to show a trend yet.