Amazon SageMaker HyperPod Global Inference Gateway: Smart routing across your ML clusters
Amazon SageMaker HyperPod just dropped the Global Inference Gateway—a managed endpoint that intelligently routes your inference requests across HyperPod clusters within a single Region or across multiple Regions. It handles failover automatically, picks the best cluster for each request, and keeps everything secure with SigV4 authentication and IAM over AWS PrivateLink.
source: [aws/machine-learning-blog]