iCreatAI News
Skip to content
Trending storyWatching

Red-Teaming Experiment: Auditors Struggle to Identify Sabotage in AI Safety Research

1 reports1 sources1 days agoUpdated

Understand this story

AI overview

Redwood Research has published a red-team/blue-team experiment: a human expert plays the role of a saboteur, carrying out an AI safety research project from scratch and is asked to substantially change the project's core conclusions; then 25 human auditors judge whether the research was sabotaged. Each auditor has 90 minutes to review the full research log and may call on frontier AI models with unlimited compute to run their own experiments, with the time spent running experiments not counted toward the 90 minutes. Redwood Research says auditors struggle to identify this kind of sabotage. The experiment also had 9 AI models serve as auditors on the same records, but they could not run experiments themselves. The saboteur knew the benign results in advance and could decide for themselves how to advance the project.

AI overview based on source reports · Updated 23 hr ago

Coverage timeline

Follow the coverage to explore different perspectives.

Nov 21
  1. Redwood Research Blog
    Auditors Struggle to Catch Sabotage in End-to-End Transcripts of AI Safety Research

    Redwood Research ran a red-team/blue-team game in which a human-expert-level saboteur produced 22 full research traces (10 benign, 12 sabotaged, containing 33 sabotage mechanisms), and 25 human auditors given 90 minutes per trace, full logs, AI-generated audits, frontier models and unlimited compute achieved only an AUROC of 0.59, not significantly better than chance. Human auditors fully identified just 17% of sabotage mechanisms and missed 74% entirely, while the strongest frontier models scored higher (Claude Fable 5.1 at 37.1% and GPT-6 Astra at 32.6% versus 21.9% for humans with partial credit) but still missed most sabotage. The authors note limitations including a single saboteur, wide confidence intervals, and the saboteur knowing benign results in advance, and caution that attack selection sharply reduced identification and that false positives would carry costs in training.

Activity for this story

Current comparable activity
5
Peak comparable activity
10Nov 21 09:00
Change in the past 24 hours
–

Trends compare the same fully observed participants, which may cover fewer participants than current activity. Move the pointer or tap the chart to inspect hourly activity; use the left and right arrow keys with a keyboard.