The Iron(wood) Throne: Google's TPU Stakes Its Claim on Inference

SemiAnalysis reports first official long-context, multi-turn agentic inference benchmark results on TPUv7 (Ironwood) across four models on InferenceX/AgentX, finding TPUv7 either competitive or best-in-class on performance per dollar for external customers, including 1.2x to 2.7x the tokens per dollar of B200, B300 and MI355X on GLM-5.2 FP8. The results came through ecosystem partners including SGLang (RadixArk), vLLM (Inferact) and llm-d, and the piece argues NVIDIA's bureaucracy and stingy cluster contributions to open-source maintainers are among the biggest risks to its moat. It also notes NVIDIA's GB300 FP4 counter submission still trails TPUv7 FP8 on end-to-end performance per dollar, while FP4 GPUs retain a per-dollar lead on GLM-5.3 until TPUv8 adds FP4 support, expected in Q1 2027.