Token benchmark of eight iOS Simulator MCP tools
Understand this story
garymiklos published a benchmark of eight iOS Simulator MCP tools, comparing the login flow on the same Mac and the same iPhone 17 Pro Simulator. Results show that simulator-use 0.15.0 reads only 194 tokens per login, 3.4 times fewer than the runner-up, 66 times fewer than Xcode MCP, and 569 times fewer than AXe. The test required each tool to have a Mac, Xcode, and iOS Simulator runtime environment; token counts are estimates (text by character count divided by 4, images by width × height divided by 750), and no Anthropic API key was set for this run. AXe and MobileBuildMCP also ran a second time with Device Hub open, but the ranking is based on the headless runs that all tools completed.
AI overview based on source reports · Updated 2 days ago
Coverage timeline
Follow the coverage to explore different perspectives.
- Hacker News AIiOS Simulator MCP benchmark: eight tools for AI agents
A benchmark on one Mac and one iPhone 17 Pro Simulator compared eight iOS Simulator tools for AI agents, each driven through its own MCP server (AXe via its command line), on a full sign-in flow. simulator-use 0.15.0 read 194 estimated tokens per sign-in, 3.4x fewer than the next tool, 66x fewer than Xcode MCP and 569x fewer than AXe, and was also fastest at 3.50 s step by step and 2.87 s batched. Every call was timed and saved, failures included, with a median of 10 runs per tool on a freshly started app.
Activity for this story
- Current comparable activity
- 3
- Peak comparable activity
- 10Nov 4 20:00
- Change in the past 24 hours
- -51%
Trends compare the same fully observed participants, which may cover fewer participants than current activity. Move the pointer or tap the chart to inspect hourly activity; use the left and right arrow keys with a keyboard.