Provisioning Networks for AI Supercomputers: A Trace-Driven Study of Performance
Provisioning Networks for AI Supercomputers: A Trace-Driven Study of Performance
This paper presents a trace-driven study of network sensitivity for training and inference workloads at scales up to 4,096 GPUs, combining signature-based trace decomposition with linear programming to enable broad network-parameter sweeps while preserving recorded NCCL schedules and program dependencies. The analysis is validated against monolithic linear programs, LogGOPSim, and hardware measurements, and yields latency-bandwidth envelopes identifying low-sensitivity regions and performance cliffs that vary across workloads. The authors state the results translate execution traces into practical guidance for evaluating network configurations, with model assumptions made explicit.
Source: Hacker News AI · arxiv.org