Amazon DynamoDB Streams introduces a new API feature for faster and more efficient stream shard discovery

Amazon DynamoDB Streams now supports a new ShardFilter parameter in the DescribeStream API to simplify and optimize the consumption of streaming data. You can use ShardFilter parameter to quickly discover child shards after a parent shard has been closed, significantly improving efficiency and responsiveness when processing data from DynamoDB Streams.

DynamoDB Streams is a serverless data streaming feature that makes it straightforward to track, process, and react to item-level changes in DynamoDB tables in near real time. DynamoDB Streams enables diverse change data capture use cases, including building event-driven applications, data replication, auditing, and implementing data analytics and machine learning capabilities. Applications consuming data from DynamoDB Streams can efficiently transition from reading a closed shard to its child shard using this optional ShardFilter parameter, avoiding repeated calls to the DescribeStream API to retrieve and traverse the shard map for all closed and open shards. This API enhancement helps ensure smoother transitions and lower latency when switching between shards, making your stream processing applications more responsive and cost-effective.

The new ShardFilter parameter is available in all AWS Regions. You can get started with the feature by using the AWS API, Kinesis Client Library (KCL) 3.0, or Apache Flink connector for DynamoDB Streams. Customers that use AWS Lambda to consume DynamoDB Streams will automatically benefit from this enhanced API experience.

For more information, refer to Working with DynamoDB Streams in the DynamoDB Developer Guide and API Reference for DescribeStream.