iCreatAI News
Skip to content
Trending storyWatching

Google achieves zero numerical difference between RL training and inference on TPUs

1 reports1 sources2 days agoUpdated

Understand this story

AI overview

Google has implemented Zero-TIM (zero training-inference difference) in the open-source JAX library Tunix, enabling the sampler and trainer to produce bitwise-identical logprobs for the same token under the same weights, thereby eliminating the numerical difference between training and inference in on-policy reinforcement learning. Google says the implementation has been validated at multi-host scale on 64 TPU v5p chips (DP8×TP8). Zero-TIM is currently a prototype in Tunix, Google's open-source library for LLM reinforcement learning and post-training on Cloud TPU. Google says it is extending support to linear attention and mixture-of-experts (MoE) architectures, meaning the existing validation scope does not yet cover these architectures.

AI overview based on source reports · Updated 1 days ago

Coverage timeline

Follow the coverage to explore different perspectives.

Nov 16
  1. Google Developers Blog
    Unlocking Stable On-Policy RL on TPU with Bitwise-Zero Train–Inference Mismatch

    Google describes a Zero train–inference mismatch (Zero-TIM) implementation in Tunix that makes the sampler and trainer assign bitwise-identical logprobs to the same action and token history, verified across 64 TPU v5p chips (DP8×TP8). The approach uses batch-invariant Pallas kernels, explicit cross-shard reduction ordering, a standardized logprob scoring tail, and custom differentiation rules for serving kernels. In a 200-step FrozenLake test with Qwen3-8B, Zero-TIM recorded zero mean absolute sampler–trainer logprob difference, while standard training and token-level truncated importance sampling retained nonzero differences.

Activity for this story

Current comparable activity
3
Peak comparable activity
6Nov 17 01:00
Change in the past 24 hours
-49%

Trends compare the same fully observed participants, which may cover fewer participants than current activity. Move the pointer or tap the chart to inspect hourly activity; use the left and right arrow keys with a keyboard.