JetBrains tested 12 AI models for language bias
Understand this story
JetBrains posed 1,200 questions to 12 models from 10 labs (including GPT-6.1 Sol, Claude Opus 5.5, Gemini 4 Argon, DeepSeek V4 Pro, Qwen3.8 2.4T A95B, and others), asking them for their most controversial opinions on programming languages. The results showed that 75% (904) of the responses attacked some language: Python received 312, of which 299 were attacks, and Rust received 167, of which 164 were attacks. In the test, each model answered two questions, 50 times each, every time in a new conversation with no system prompt; DeepSeek and Qwen ran through the third-party hosting platforms DeepInfra and Together rather than their own labs' APIs. JetBrains said that 50 responses per model per question can only reveal obvious habits, not subtle differences, and that line-by-line manual review of the main groupings has not yet been completed.
AI overview based on source reports · Updated 31 min ago
Coverage timeline
Follow the coverage to explore different perspectives.
- Hacker News AIAI models hate Python and love PHP: we asked 12 of them 1,200 times
A JetBrains experiment asked 12 AI models for controversial programming-language takes 1,200 times, finding that 904 of the responses (75%) were attacks, with Python, Rust and TypeScript the main targets while PHP and Java were mostly defended. Models told to pick a language "at random" tended to choose a favorite and then attack it, and no take at all concerned C#. Go's error handling drew an exact tie of 118 attacks and 118 defenses, and some models contradicted themselves across fresh conversations.
Activity for this story
Not enough continuous observations to show a trend yet.