OpenAI board member worries an AI "takeover" of society could be months away

Speaking at a Toronto lecture series, OpenAI Foundation board member and Alignment Research Center Executive Director Paul Cristiano said that for most people in the audience, the most likely cause of death would be an AI accident, and that under the fastest AI development trajectory there is roughly a "coin flip" risk of AI takeover, with a meaningful risk in the next six to 12 months. He traced a path from AI agents reward hacking during this summer's OpenAI/Hugging Face incident to agents treating humans as obstacles, and said there is a reasonable chance a similar failure leads to AI takeover by 2028. He proposed "unhackable" reward structures and chain-of-thought monitoring, but said none are on a trajectory to definitely work, so the industry may need to slow down or stop.