SageMaker HyperPod launches cross-region inference gateway
Understand this story
AWS releases Amazon SageMaker HyperPod Global Inference Gateway, which uses a single SageMaker managed endpoint to route inference requests to any registered HyperPod cluster, supporting same-region and cross-region calls. The feature is available in 13 AWS regions, including US East (N. Virginia), US East (Ohio), US West (N. California), US West (Oregon), South America (São Paulo), Europe (Frankfurt), Europe (Ireland), Europe (London), Europe (Stockholm), and Asia Pacific (Tokyo), Asia Pacific (Mumbai), Asia Pacific (Singapore), Asia Pacific (Sydney). In use, a cluster must first be published before it can be accessed by the gateway; if an unpublished cluster is listed in a gateway, calls will fail with a prompt indicating what needs to be corrected. Clusters that are published but not added to any gateway will not receive traffic, but their network load balancers still run and are billed.
AI overview based on source reports · Updated 1 days ago
Coverage timeline
Follow the coverage to explore different perspectives.
- AWS Machine Learning BlogIntroducing Amazon SageMaker HyperPod Global Inference Gateway: One endpoint across Regions and clusters
Amazon SageMaker HyperPod introduces the Global Inference Gateway, a SageMaker-managed endpoint that routes requests across every registered HyperPod cluster in one Region or across Regions, with automatic failover and SigV4/IAM authentication, while each cluster's existing Inference Gateway still selects the pod. The gateway scores healthy clusters on KV cache headroom, queue depth, Region proximity, and error rate, with configurable weights, and carries traffic over AWS PrivateLink. It is available in 13 AWS Regions.
- AWS Machine Learning BlogAmazon SageMaker Inference: 2026 launches in review, part 2
This second part of Amazon's 2026 SageMaker Inference review covers 13 additional launches, bringing the year's total to 26, spanning managed endpoints and HyperPod Inference. Updates include inference recommendations in SageMaker AI Studio, the Python SDK v3, LoRA adapter support, SLA-based concurrency search, an agent skill, Inference Components in the console, NVIDIA MIG GPU partitioning, managed NVIDIA Dynamo, KV-cache-aware routing, the Inference Gateway and Global Inference Gateway, Cross-Region Inference Routing, and continuous performance monitoring. The post follows a hypothetical support-assistant team through five stages, from choosing configurations to routing requests across Regions and keeping endpoints tuned after launch.
Activity for this story
- Current comparable activity
- 5
- Peak comparable activity
- 10Nov 21 07:00
- Change in the past 24 hours
- -51%
Trends compare the same fully observed participants, which may cover fewer participants than current activity. Move the pointer or tap the chart to inspect hourly activity; use the left and right arrow keys with a keyboard.