Amazon SageMaker HyperPod Inference Gets NVIDIA Dynamo Support for Massive Models
Amazon SageMaker HyperPod Inference now supports NVIDIA Dynamo, letting you run models too big for a single GPU across multiple nodes without building your own distributed infrastructure. The setup handles long prompts smoothly by splitting prefill and decode stages into separate GPU pools, plus uses smart KV-aware routing to cut recomputation and boost throughput.
source: [aws/whats-new]