Announcing enhanced autoscaling for Amazon OpenSearch Ingestion pipelines

Amazon OpenSearch Ingestion now supports enhanced autoscaling capabilities, allowing pipelines to scale dynamically based on additional parameters, including Amazon SQS queue size, persistent buffer lag, and the number of incoming HTTP connections. These enhancements improves upon the existing scaling mechanism, which previously relied only on memory and CPU utilization, providing a more comprehensive and responsive scaling mechanism for your data ingestion workloads.

With these improvements, customers can build more resilient and efficient data ingestion pipelines that automatically adapt to varying workloads. The new autoscaling parameters help optimize resource utilization, reduce ingestion bottlenecks, and improve overall pipeline performance, making it easier to handle high-throughput data streams for log analytics, observability, and security analytics use cases.

The enhanced autoscaling capabilities are now available in all AWS Regions where Amazon OpenSearch Ingestion is currently offered. You can take advantage of these improvements by updating your existing pipelines or creating new pipelines through the Amazon OpenSearch Service console or APIs at no additional cost.

To learn more, see the Amazon OpenSearch Ingestion webpage and the Amazon OpenSearch Service Developer Guide.