April 2, 2020

Amazon MSK adds support for Apache Kafka version 2.4.1

Amazon Managed Streaming for Apache Kafka (Amazon MSK) now supports Apache Kafka version 2.4.1 for new clusters. Apache Kafka 2.4.1 includes several bug fixes and new features that reduce latency and improve performance. Support for Apache Kafka version 2.4.1 is offered in all regions where Amazon MSK is available. For a complete list of improvements and bug fixes, see the Apache Kafka release notes for 2.4.1 and 2.4.0.

A few key features included in 2.4.1 are:

KIP-392, Allow Consumers to Fetch from Closest Replica, this allows consumers to fetch data from the nearest replica partition rather than from the leader partition, which reduces latency and potential cross Availability Zone data transfer.
KIP-480, Sticky Partitioner, this provides performance improvements to the default Apache Kafka partitioner with up to 50% reduction in latency.
KIP-455, Create an Administrative API for Replica Reassignment Apache, this provides a new Apache Kafka Admin API for partition reassignment and addresses limitations of the Apache ZooKeeper implementation.

Amazon MSK is a fully managed service for Apache Kafka that makes it easy for you to build and run applications that use Apache Kafka as a data store. Amazon MSK is 100% compatible with Apache Kafka, which enables you to quickly migrate your existing Apache Kafka workloads to Amazon MSK with confidence or build new ones from scratch. With Amazon MSK, you spend more time innovating on applications and less time managing clusters. To learn how to get started, see the Amazon MSK Developer Guide.

Recent Announcements
Amazon MSK can now stream broker logs to CloudWatch Logs, Amazon S3, or Amazon Elasticsearch Service
Amazon MSK increases the default broker limit per cluster to 30 brokers
Amazon MSK is now available in the AWS Middle East (Bahrain) Region
Amazon MSK now supports Apache Kafka 2.3.1
Monitor your Amazon MSK clusters using Open Monitoring with Prometheus Run fully managed Apache Flink applications with Amazon MSK
Dynamically expand your Amazon MSK clusters

April 2, 2020

Amazon MSK adds support for Apache Kafka version 2.4.1

Back to feed View original

A few key features included in 2.4.1 are:

KIP-392, Allow Consumers to Fetch from Closest Replica, this allows consumers to fetch data from the nearest replica partition rather than from the leader partition, which reduces latency and potential cross Availability Zone data transfer.
KIP-480, Sticky Partitioner, this provides performance improvements to the default Apache Kafka partitioner with up to 50% reduction in latency.
KIP-455, Create an Administrative API for Replica Reassignment Apache, this provides a new Apache Kafka Admin API for partition reassignment and addresses limitations of the Apache ZooKeeper implementation.