OpenAI text watermarking struggles to meet new EU rules
Understand this story
IEEE Spectrum reports that testing of textGrain, OpenAI's text watermarking released in October and intended to comply with the EU AI Act, shows that the watermark often fails to persist when text is edited again or is shorter than 150 words. The relevant EU rules do not require watermarking outputs shorter than 200 tokens. OpenAI is working with a limited research group including Cornell University assistant professor John Thickstun and researchers from Slovakia's KInIT; Thickstun and KInIT senior researcher Dominik Macko said they had only just been granted access and had not yet begun evaluating the system.
AI overview based on source reports · Updated 20 min ago
Coverage timeline
Follow the coverage to explore different perspectives.
- Hacker News AIAI Text Watermarking May Not Be Able to Meet the E.U.'S New Rules
Tests of OpenAI's textGrain watermark, released in October to comply with the E.U. AI Act's transparency obligations, show detection falls to about 80% for 200-token texts versus 95% for 400-token texts, and replacing 25% of words with synonyms cuts detection from 92% to 17%. The E.U. Code of Practice does not require watermarks on outputs shorter than 200 tokens, and with no measurable legal thresholds and no peer-reviewed evaluation yet, it is too early to say whether textGrain meets the requirements. OpenAI says textGrain matched or outperformed Google's SynthID Text in its own unverified tests, and researchers including Cornell's John Thickstun and KInIT's Dominik Macko have only just gained access to evaluate it.
Activity for this story
Not enough continuous observations to show a trend yet.