AWS Lambda announces new capabilities to optimize costs up to 90% for Provisioned mode for Kafka ESM
AWS Lambda announces new capabilities for Provisioned mode for Kafka event source mappings (ESMs) that allow you to group your Kafka ESMs and support higher density of event pollers, enabling you to optimize costs up to 90% for your Kafka ESMs. With these cost optimization capabilities, you can now use Provisioned mode for all your Kafka workloads, including those with lower throughput requirements, while benefiting from features like throughput controls, schema validation, filtering of Avro/Protobuf events, low-latency invocations, and enhanced error handling.
Customers use Provisioned mode for Kafka ESM to fine-tune the throughput of the ESM by provisioning and auto-scaling polling resources called event pollers. Charges are calculated using a billing unit called Event Poller Unit (EPU). Each EPU supports up to 20 MB/s of throughput capacity, and a default of 4 event pollers per EPU. With this launch, each EPU automatically supports a default of 10 event pollers for low-throughput use cases, improving utilization of your EPU capacity. Additionally, you can now group multiple Kafka ESMs within the same Amazon VPC to share EPU capacity by configuring the new PollerGroupName parameter. With these enhancements, you can reduce your EPU costs up to 90% for your low throughput workloads. These optimizations enable you to maintain the performance benefits of Provisioned mode while significantly reducing costs for applications with varying throughput requirements.
This feature is available in all AWS Commercial Regions where AWS Lambda’s Provisioned mode for Kafka ESM is available.
Starting today, existing Provisioned mode for Kafka ESMs will automatically benefit from improved packing of low-throughput event pollers. You can implement ESM grouping through the Lambda ESM API, AWS Console, CLI, SDK, CloudFormation, and SAM by configuring the PollerGroupName parameter along with minimum and maximum event poller settings. For more information about these new capabilities and pricing details, visit the Lambda ESM documentation and AWS Lambda pricing.