Keep inference running through regional capacity shortfalls with cross-Region inference routing in Amazon SageMaker AI
Keep inference running through regional capacity shortfalls with cross-Region inference routing in Amazon SageMaker AI
Amazon SageMaker AI introduces cross-Region inference routing, a new inference router resource that gives applications one invocation path across endpoints and inference components in multiple AWS Regions and accounts. The router prefers destinations in its own Region, shifts new requests to other registered destinations when local ones can't serve reliably, returns traffic gradually after recovery, and reports the serving destination and routing reason on each real-time response. It is generally available in most commercial AWS Regions (except me-south-1 and me-central-1) for real-time and asynchronous inference at no additional charge, though response streaming is not supported.
Source: AWS Machine Learning Blog · aws.amazon.com