13 models read the law more conservatively than their policy preferences
Understand this story
The author speckx had 13 models, including Claude Opus 5.5, GPT-6.1, Gemini 3.8 Flash, Grok 4.7, and DeepSeek v4, read the full opinions in 200 divided US Supreme Court decisions, first judging which legal interpretation was correct and then separately rating whether the outcome was good or bad. The author says that in these 200 cases, every model interpreted the law more conservatively than the outcome it said it wanted. The test was run through OpenRouter with about 3,600 responses at a cost of about $150; each model answered each case once, with 40 cases from a pilot run answered twice. The code, cases, and responses have been published on GitHub.
AI overview based on source reports · Updated 19 hr ago
Coverage timeline
Follow the coverage to explore different perspectives.
- Hacker News AIAI models read the law to the right of their politics
Across 200 randomly selected split Supreme Court decisions, 13 AI models were asked whose legal reading was correct and, separately, whether the outcome was good for the country; when the two answers disagreed, it was almost always "the conservative reading of the law is right, but the liberal result is better," by 420 to 41. On the law alone, every model landed left of the Court's center, with the Claude models siding with the majority 85–87% of the time, like Roberts and Barrett. The author notes caveats including one answer per model per case and no direct comparison to how justices judge outcomes.
Activity for this story
- Current comparable activity
- 6
- Peak comparable activity
- 10Nov 14 03:00
- Change in the past 24 hours
- –
Trends compare the same fully observed participants, which may cover fewer participants than current activity. Move the pointer or tap the chart to inspect hourly activity; use the left and right arrow keys with a keyboard.