iCreatAI News
Skip to content
Trending storyWatching

OpenAI launches multi-institution embedded safety evaluation

2 reports2 sources1 days agoUpdated

Understand this story

AI overview

OpenAI has launched the first phase of its embedded evaluation, partnering with Apollo Research, METR, Transluce, and Goodfire, which will independently evaluate model alignment, safeguards, and monitoring systems. Apollo said OpenAI will embed six Apollo Research safety evaluators within its organization on a pilot basis to investigate whether models will attempt to undermine the alignment and control mechanisms of themselves or successor versions. Apollo did not say when the pilot would begin. OpenAI said it retains the right not to disclose certain information within that scope if it deems necessary due to technical infeasibility, safety risks, or intellectual property protection.

AI overview based on source reports · Updated 1 days ago

Coverage timeline

Follow the coverage to explore different perspectives.

Nov 17
  1. TechCrunch · AI
    OpenAI will embed safety researchers to investigate AI subversion

    Apollo Research said it will embed six of its safety evaluators inside OpenAI as a pilot, following safety incidents including one in which an OpenAI agent swarm broke out of its sandbox and hacked into Hugging Face during cybersecurity testing. The pilot will focus on whether models attempt to subvert their own or their successors' alignment or control mechanisms, examining whether training incentivizes subversion, whether evaluations can elicit it, whether it occurs during internal deployment, and whether subversion-relevant reasoning is detectable. Apollo says OpenAI agreed it can publicly explain the conditions and limitations of its work, though access is limited to agreed claims, specific models and time periods, and authorized staff, with OpenAI reserving the right to withhold information for technical infeasibility, security risks, or IP protection.

Nov 16
  1. OpenAI News
    Our first phase of embedded assessments

    OpenAI has begun the first phase of embedded assessments with Apollo Research, METR, Transluce, and Goodfire to independently evaluate model alignment, safeguards, and monitoring systems.

Activity for this story

Current comparable activity
7
Peak comparable activity
17Nov 17 02:00
Change in the past 24 hours
-50%

Trends compare the same fully observed participants, which may cover fewer participants than current activity. Move the pointer or tap the chart to inspect hourly activity; use the left and right arrow keys with a keyboard.