iCreatAI News
Skip to content
AWS Machine Learning Blog· Dmitry Soldatkin·· 1 days agoAI score58

Introducing Amazon SageMaker HyperPod Global Inference Gateway: One endpoint across Regions and clusters

Introducing Amazon SageMaker HyperPod Global Inference Gateway: One endpoint across Regions and clusters

AI summary

Amazon SageMaker HyperPod introduces the Global Inference Gateway, a SageMaker-managed endpoint that routes requests across every registered HyperPod cluster in one Region or across Regions, with automatic failover and SigV4/IAM authentication, while each cluster's existing Inference Gateway still selects the pod. The gateway scores healthy clusters on KV cache headroom, queue depth, Region proximity, and error rate, with configurable weights, and carries traffic over AWS PrivateLink. It is available in 13 AWS Regions.

Source: AWS Machine Learning Blog · aws.amazon.com