AWS just dropped Agent Skills for Amazon MSK that handle the boring stuff—diagnosing performance hiccups, right-sizing your clusters with cost breakdowns, and planning migrations from self-managed Kafka to Amazon MSK. Think of it as having a Kafka expert in your pocket who knows exactly what your broker needs.
Amazon EMR Serverless now packs 32 vCPU / 244 GB worker instances that slash query times by 29% and cut costs equally on demanding Spark workloads. The real win? Shuffle-heavy joins and multi-table queries finally get the horsepower they deserve, backed by solid TPC-DS and TPC-H benchmark results.
AWS Spark Upgrade Agent automates the painful migration of Apache Spark 3.x to 4.0 on Amazon EMR, handling API deprecations, behavioral changes, and build configs so you don't have to manually grind through months of work. The tool validates your jobs and turns what used to take forever into just a few hours.
Amazon EMR now leverages Apache Celeborn to optimize remote shuffle operations across EKS and EC2 deployments, delivering better job reliability without breaking the bank. This deep dive shows how the service pipeline handles data shuffling more efficiently, giving you the performance gains and cost savings you've been waiting for.