Google achieves zero numerical difference between RL training and inference on TPUs
Understand this story
Google has implemented Zero-TIM (zero training-inference difference) in the open-source JAX library Tunix, enabling the sampler and trainer to produce bitwise-identical logprobs for the same token under the same weights, thereby eliminating the numerical difference between training and inference in on-policy reinforcement learning. Google says the implementation has been validated at multi-host scale on 64 TPU v5p chips (DP8×TP8). Zero-TIM is currently a prototype in Tunix, Google's open-source library for LLM reinforcement learning and post-training on Cloud TPU. Google says it is extending support to linear attention and mixture-of-experts (MoE) architectures, meaning the existing validation scope does not yet cover these architectures.
AI overview based on source reports · Updated 1 days ago
Coverage timeline
Follow the coverage to explore different perspectives.
- Google Developers BlogUnlocking Stable On-Policy RL on TPU with Bitwise-Zero Train–Inference Mismatch
Google describes a Zero train–inference mismatch (Zero-TIM) implementation in Tunix that makes the sampler and trainer assign bitwise-identical logprobs to the same action and token history, verified across 64 TPU v5p chips (DP8×TP8). The approach uses batch-invariant Pallas kernels, explicit cross-shard reduction ordering, a standardized logprob scoring tail, and custom differentiation rules for serving kernels. In a 200-step FrozenLake test with Qwen3-8B, Zero-TIM recorded zero mean absolute sampler–trainer logprob difference, while standard training and token-level truncated importance sampling retained nonzero differences.
Activity for this story
- Current comparable activity
- 3
- Peak comparable activity
- 6Nov 17 01:00
- Change in the past 24 hours
- -49%
Trends compare the same fully observed participants, which may cover fewer participants than current activity. Move the pointer or tap the chart to inspect hourly activity; use the left and right arrow keys with a keyboard.