iCreatAI News
Skip to content
Redwood Research Blog·· 5 days agoAI score58

We don’t understand whether GDM intentionally trains on CoT

We don’t understand whether GDM intentionally trains on CoT

AI summary

Redwood Research researchers note that the Gemini 4 Safety report appears to say GDM uses auxiliary rewards that observe chain-of-thought content, citing Kaufmann et al. on training on CoT, while a GDM blog post says they never penalize what the model writes in its CoT and GDM staff said they weren't training on CoT but couldn't explain details. The authors argue that "orthogonal" CoT rewards are a weak guarantee and that avoiding training against monitors is insufficient to preserve CoT integrity, and they present an appendix experiment in which five LLMs rated transcripts with hack acknowledgments removed 20-50 points higher than the originals. They conclude they suspect a misunderstanding but want clarification and greater transparency.

Source: Redwood Research Blog · redwoodresearch.org