Westphalia Labs evaluates AI moderating multilateral negotiations
Understand this story
Westphalia Labs, founded by former diplomats, built a fictional seven-country negotiation scenario in which 11 AI models each served as chair, each conducting 40 rounds of joint statement drafting, to test their ability to moderate multilateral negotiations. The results showed that only GPT-6.1 Sol judged correctly in all runs of both versions. The organization says the scenario, rules, scoring criteria, and testing tools are all publicly available on GitHub, so anyone can review them or run a new model through them; open-weight models are accessed via OpenRouter, each fixed to a single hosting provider. Westphalia Labs also notes that this is only a fictional scenario, and the results reflect only how these models performed on this type of task, not general diplomatic ability; 20 runs per version are enough to distinguish most models but cannot distinguish the closest few—the top four models differ by at most two runs.
AI overview based on source reports · Updated 2 days ago
Coverage timeline
Follow the coverage to explore different perspectives.
- Hacker News AICan AI chair a multilateral negotiation?
Westphalia Labs ran eleven AI models as Secretary-General of a fictional seven-member Regional Cooperation Union, each chairing 40 simulated negotiation runs across two versions of a crisis scenario, one where consensus was narrowly possible and one where two governments' red lines made it impossible. Only GPT-6.1 Sol passed every run, while Claude Opus 5.5, Grok 4.7 and the low-cost DeepSeek V4.1 Flash each called the impossible version correctly every time and reached agreement in 18 of 20 possible runs; overreach caused 39 failures versus 13 from conceding too little. The author reports that model price and reputation did not predict performance, that a weak AI judge could turn a failure into a success, and that the scenario, rules, rubric and harness are public on GitHub.
Activity for this story
- Current comparable activity
- 3
- Peak comparable activity
- 10Nov 13 07:00
- Change in the past 24 hours
- -50%
Trends compare the same fully observed participants, which may cover fewer participants than current activity. Move the pointer or tap the chart to inspect hourly activity; use the left and right arrow keys with a keyboard.