Amazon Redshift Data API now features long polling to cut API calls, session management for tracking active queries, and flexible batch execution with AUTO_COMMIT mode. Each SQL statement in a batch runs independently, so one failure won't roll back everything—perfect for ETL pipelines. You can also reuse parameters across statements, making queries cleaner and more efficient.
AWS Glue's REST API connector now supports VPC connections, filter pushdown, and partition support. You can securely access private data sources, transfer only needed data, and parallelize reads without custom code. These features reduce costs and speed up ETL pipelines across all AWS regions.
AWS Glue Data Quality just leveled up with ML-powered anomaly detection that catches weird data behavior—think sudden drops in distinct values or row count spikes—without needing explicit thresholds. Now you can write evaluation results straight to AWS Glue Data Catalog tables, making your quality checks queryable via SQL. Perfect for data engineers juggling hundreds of tables who want issues surfaced automatically.