AWS Glue jobs can now include AWS Glue DataBrew Recipes

AWS Glue Studio Visual ETL jobs now let you use DataBrew Recipes as steps in the flow of transformations. AWS Glue Studio Visual ETL is a no-code job authoring UI for ETL developers with a flow-diagram interface. AWS Glue DataBrew is a no code data preparation tool for business users and data analysts with a spreadsheet-style UI. The new integration between the two makes it simpler to deploy and scale DataBrew jobs and gives DataBrew users access to AWS Glue features not available in DataBrew. The integration also works in code-based jobs.

After business users define data cleansing and preparation “recipes” in DataBrew, technical users often deploy and scale up DataBrew jobs. With this new integration, customers can now scale up recipes to process petabytes of data, and at the lower price point for AWS Glue jobs. You can run DataBrew recipes in AWS Glue 4.0, which offers better performance and auto-scaling for reduced costs. It also lets you use recipes with AWS Glue features including custom transforms, incremental processing, and combining multiple datasets.

This feature is available in the same AWS Regions as AWS Glue, except the Asia Pacific (Osaka), Middle East (Bahrain), GovCloud, and China regions.

To learn more, visit our documentation and read the blog post.

7/31/2023 - This post has been updated to correct the list of regions where the feature is available.