Test: Gemini Decision 1 struggles to distinguish new AI speech
Understand this story
Author maplet posted test results on Hacker News stating that Gemini Decision 1's ability to distinguish AI-generated speech from real human speech depends on the release date of the speech generator. The test used 30 audio clips each from 34 speech generators in MLAAD, compared against 200 real human readings from LibriTTS: for generators released in 2023–2024, the discrimination AUC was 0.67; for generators released in 2025–2026, it was only 0.52, close to random guessing. This means the model is essentially unable to reliably identify AI speech produced by newer generators.
AI overview based on source reports · Updated 1 days ago
Coverage timeline
Follow the coverage to explore different perspectives.
- Hacker News AIGemini Decision 1 can't separate AI voices made after 2024
Testing Gemini Decision 1 against 30 clips each from 34 AI voice generators and 200 human LibriTTS audiobook clips, the author found pooled ROC-AUC of 0.67 for 2017–2024 generators but only 0.52, effectively chance, for 2025–2026 generators. The model also showed a strong bias toward labeling audio human-made, giving only 20 of 1,020 AI clips a 50% or higher chance of being AI-generated, and scored just 0.58 on Gemini 3.1 Flash TTS. Clips were re-encoded with ffmpeg to strip metadata and band-limited to 8 kHz before scoring.
Activity for this story
- Current comparable activity
- 5
- Peak comparable activity
- 10Nov 17 10:00
- Change in the past 24 hours
- –
Trends compare the same fully observed participants, which may cover fewer participants than current activity. Move the pointer or tap the chart to inspect hourly activity; use the left and right arrow keys with a keyboard.