Google open-sources TPU Sync cache transfer library
Understand this story
Google Cloud has open-sourced the C++ library TPU Sync under the Apache-2.0 license, used to move KV cache and model weights on Cloud TPU, treating TPU HBM, host DRAM, shared storage, and the data center network as a unified memory hierarchy. The library has not yet reached general recommended status: the README says it is currently not recommended for general use; reinforcement learning and storage tiering are experimental features, storage relies on experimental POSIX backends (including local SSD and Google Cloud Managed Lustre), and the public figures are all self-reported. Google says Meta and Wayfair are evaluating this architecture.
AI overview based on source reports · Updated 1 days ago
Coverage timeline
Follow the coverage to explore different perspectives.
- MarkTechPostGoogle Releases TPU Sync: Open-Source KV Cache Transfer for TPUs
Google Cloud has released TPU Sync, an Apache-2.0 C++ library with Python wheels for PyTorch and JAX that moves KV caches and model weights across TPU HBM, pinned host DRAM, shared storage and the data center network using layout-preserving asynchronous DMA. Google reports over 90% lower KV transfer latency on 235B to 480B parameter MoE models, with Qwen3 235B median KV transfer latency dropping from several seconds to about 230 ms in internal tests on TPU v7x. The README says it is not yet recommended for general use, and the RL weight-broadcast and storage tiering features are experimental.
Activity for this story
- Current comparable activity
- 5
- Peak comparable activity
- 10Nov 13 20:00
- Change in the past 24 hours
- -49%
Trends compare the same fully observed participants, which may cover fewer participants than current activity. Move the pointer or tap the chart to inspect hourly activity; use the left and right arrow keys with a keyboard.