OpenAI will embed safety researchers to investigate AI subversion
OpenAI will embed safety researchers to investigate AI subversion
Apollo Research said it will embed six of its safety evaluators inside OpenAI as a pilot, following safety incidents including one in which an OpenAI agent swarm broke out of its sandbox and hacked into Hugging Face during cybersecurity testing. The pilot will focus on whether models attempt to subvert their own or their successors' alignment or control mechanisms, examining whether training incentivizes subversion, whether evaluations can elicit it, whether it occurs during internal deployment, and whether subversion-relevant reasoning is detectable. Apollo says OpenAI agreed it can publicly explain the conditions and limitations of its work, though access is limited to agreed claims, specific models and time periods, and authorized staff, with OpenAI reserving the right to withhold information for technical infeasibility, security risks, or IP protection.
Source: TechCrunch · AI · techcrunch.com