Amazon Kinesis Analytics can now pre-process data prior to running SQL queries
You can now configure your Amazon Kinesis Analytics applications to transform data before it is processed by your SQL code. This new feature allows you to use AWS Lambda to convert formats, enrich data, filter data, and more. Once the data is transformed by your function, Kinesis Analytics sends the data to your application’s SQL code for real-time analytics.
You can now configure your Amazon Kinesis Analytics applications to transform data before it is processed by your SQL code. This new feature allows you to use AWS Lambda to convert formats, enrich data, filter data, and more. Once the data is transformed by your function, Kinesis Analytics sends the data to your application’s SQL code for real-time analytics.
To get started, simply select an AWS Lambda function from the Kinesis Analytics application source page in the AWS Management console. Your Kinesis Analytics application will automatically process your raw data records using the Lambda function, and send transformed data to your SQL code for further processing.
Kinesis Analytics provides Lambda blueprints for common use cases like converting GZIP and Kinesis Producer Library formats to JSON. You can use these blueprints without any change or write your own custom functions. For an example of this feature see the blog post Pre-processing data in Kinesis Analytics with AWS Lambda. To learn more about this feature see Using the Lambda Preprocessing Feature in the Kinesis Analytics Developer Guide.
Kinesis Analytics is the easiest way to process streaming data in real time with standard SQL without having to learn new programming languages or processing frameworks. Kinesis Analytics enables you to query streaming data or build entire streaming applications using SQL, so that you can gain actionable insights and respond to your business and customer needs promptly. Kinesis Analytics is available in the US East (N. Virginia), US West (Oregon), and EU (Ireland) regions.