AWS Glue's REST API connector now supports VPC connections, filter pushdown, and partition support. You can securely access private data sources, transfer only needed data, and parallelize reads without custom code. These features reduce costs and speed up ETL pipelines across all AWS regions.
AWS Glue Data Quality just leveled up with ML-powered anomaly detection that catches weird data behavior—think sudden drops in distinct values or row count spikes—without needing explicit thresholds. Now you can write evaluation results straight to AWS Glue Data Catalog tables, making your quality checks queryable via SQL. Perfect for data engineers juggling hundreds of tables who want issues surfaced automatically.
AWS Glue Data Quality just dropped a Distribution Analyzer that generates frequency profiles for your data—think histograms for numbers and value distributions for categories. You can now spot skewness, outliers, and weird patterns without writing custom code, and results get stored in S3 for querying via Athena. It integrates seamlessly with your existing DQDL rulesets, so profiling runs alongside your current checks in one go.