Amazon ElastiCache for Valkey Adds Distributed KV Caching for Faster LLM Inference
Amazon ElastiCache for Valkey now supports distributed KV caching, slashing self-hosted LLM inference costs by up to 40% and time-to-first-token by up to 80%. The feature is available across 13 AWS regions in the US, Europe, and Asia Pacific, and integrates with popular inference engines like vLLM, SGLang, and TensorRT-LLM through LMCache.
source: [aws/whats-new]