Reviewing My 2017 AI Risk Project Including with Dario at OpenAI

The author revisits a 2017 series of twelve blog posts on AI risk, including conversations with Dario Amodei, Michael Littman, and others, concluding he talked to the right people and heard prescient ideas but got the bottom line wrong. He notes that prosaic AGI, which many mainstream ML researchers then thought impossible, largely happened through scaling, and that Concrete Problems in AI Safety held up better than Highly Reliable Agent Design. He says he never asked Amodei whether AI reliably doing what we want might become a limiting factor on AI progress, potentially accelerating development along riskier short timelines.