SageMaker AI cross-region inference routing is generally available
Understand this story
Amazon SageMaker AI's cross-region inference routing is now generally available. AWS said the feature has been offered in most commercial Regions since November 18, 2026, except me-south-1 and me-central-1; it supports real-time and asynchronous inference, whereas earlier descriptions mentioned only real-time endpoints. The feature uses a new inference router resource to let applications cover running endpoints and inference components across multiple AWS regions and accounts with a single invocation path. Customers can set the per-instance request rate and the maximum share of endpoint capacity that remote traffic may occupy in the receiving endpoint's CrossRegionTrafficConfig; local traffic does not count toward that limit. AWS said there is no additional charge for using the router, and endpoints and inference components are billed at standard SageMaker AI prices; routing metrics are published to CloudWatch and billed at standard rates. Salesforce's Agentforce Foundation team has adopted the feature to provide a unified invocation path for models distributed across more than 30 global Regions, judging health based on each endpoint's recent request results and typically beginning to shift traffic to healthy endpoints within about a minute.
AI overview based on source reports · Updated 17 hr ago
Coverage timeline
Follow the coverage to explore different perspectives.
- AWS Machine Learning BlogAmazon SageMaker Inference: 2026 launches in review, part 2
This second part of Amazon's 2026 SageMaker Inference review walks through 13 launches (numbered 14–23) covering inference recommendations in SageMaker AI Studio, the SageMaker Python SDK v3, and coding agents, plus LoRA adapter support and SLA-based concurrency search. It also details Inference Components in the console, NVIDIA MIG GPU partitioning, managed NVIDIA Dynamo on HyperPod Inference, KV-cache-aware routing, the HyperPod Inference Gateway, and Cross-Region Inference Routing. The post closes with continuous performance monitoring and in-place rolling model updates, illustrated through a hypothetical support-assistant team scaling to more models and Regions.
- AWS Machine Learning BlogKeep inference running through regional capacity shortfalls with cross-Region inference routing in Amazon SageMaker AI
Amazon SageMaker AI introduces cross-Region inference routing, a new inference router resource that gives applications one invocation path across endpoints and inference components in multiple AWS Regions and accounts. The router prefers destinations in its own Region, shifts new requests to other registered destinations when local ones can't serve reliably, returns traffic gradually after recovery, and reports the serving destination and routing reason on each real-time response. It is generally available in most commercial AWS Regions (except me-south-1 and me-central-1) for real-time and asynchronous inference at no additional charge, though response streaming is not supported.
- AWS Machine Learning BlogHow Salesforce is advancing global-scale model serving with cross-Region inference routing in Amazon SageMaker AI
Salesforce's Agentforce Foundation team serves in-house and open source models across 30+ Regions and hundreds of AWS accounts, where per-Region hosting fragmented capacity and made traffic spikes an availability problem. Using SageMaker AI's inference router, Salesforce replaced its Internal Gateway's self-built health checks, retries and Region-switching logic with a managed routing layer that prefers the local Region, shifts traffic on 5xx errors, throttles and timeouts, and keeps data-residency-tagged requests pinned in-Region. In a failover test, traffic began moving to a remote Region about two minutes after the first errors, peaked at roughly 78 percent, and returned gradually over about 10 minutes, with cross-Region latency adding under 100 milliseconds at p50.
Activity for this story
- Current comparable activity
- 6
- Peak comparable activity
- 10Nov 20 03:00
- Change in the past 24 hours
- -5%
Trends compare the same fully observed participants, which may cover fewer participants than current activity. Move the pointer or tap the chart to inspect hourly activity; use the left and right arrow keys with a keyboard.