Spark Connect on Amazon EMR on EKS: Code Anywhere, Run Everywhere
Amazon just dropped Spark Connect on EMR on EKS, letting you develop PySpark interactively from VS Code, PyCharm, Jupyter, SageMaker Unified Studio, or dbt while your heavy lifting runs on existing EKS clusters. No more context switching—write and debug locally, execute at scale remotely.
source: [aws/big-data-blog]