Google open-sources TPU training-inference numerical alignment implementation
Understand this story
Google has open-sourced the Zero-TIM implementation in Tunix, aiming to eliminate numerical discrepancies between reinforcement learning training and inference: under identical weights, the sampler decoding, serving prefill, and trainer forward pass give bitwise-identical logprobs for the same action. The implementation is based on JAX/XLA and Pallas and targets TPUs. Google says it has validated this on a multi-host mesh of 64 TPU v5p chips (DP8×TP8). Support is still limited, and Google says it is expanding support for linear attention and mixture-of-experts (MoE) architectures.
AI overview based on source reports · Updated 1 hr ago
Coverage timeline
Follow the coverage to explore different perspectives.
- Google Developers BlogUnlocking Stable On-Policy RL on TPU with Bitwise-Zero Train–Inference Mismatch
Google describes a Zero train–inference mismatch (Zero-TIM) implementation in Tunix that makes sampler decode, serving prefill and trainer forward assign bitwise-identical logprobs to the same sampled action under the same weights, verified across a 64-chip TPU v5p (DP8×TP8) mesh. The recipe combines batch-invariant Pallas kernels with fixed accumulation order, explicit cross-shard reduction ordering, a standardized logprob scoring tail, and custom VJP rules that reuse serving kernels for forward evaluation. In a 200-step FrozenLake test with Qwen3-8B, Zero-TIM recorded zero mean absolute sampler–trainer logprob difference at every logged step, while standard training and token-level truncated importance sampling retained nonzero differences.
Activity for this story
Not enough continuous observations to show a trend yet.