What is Pathways? Single-controller training for large models

Google announces the General Availability of Pathways on Google Cloud Platform, describing the distributed system it built to train large models including Gemini. The post explains where SPMD execution falls short—pipeline bubbles, RL loops with distinct programs, and failures that stop every process—and how Pathways instead has a single Python client declare the whole computation as a graph of blocks and tensors, with a distributed runtime placing it on virtual devices, moving and resharding tensors, ordering work across the cluster, and remapping around a failed host. It also contrasts Pathways with Ray, framing Ray as orchestration and Pathways as the execution layer beneath it.