AWS Glue now provides workflows to orchestrate your ETL workloads
Starting today, you can now use workflows in AWS Glue to author directed acyclic graphs (DAGs) of Glue triggers, crawlers and jobs.
Starting today, you can now use workflows in AWS Glue to author directed acyclic graphs (DAGs) of Glue triggers, crawlers and jobs.
Workflows enable orchestration of your ETL workloads by building dependencies between Glue entities (triggers, crawlers and jobs). You can visually track status of the different nodes in the workflows on the console making it easier to monitor progress and troubleshoot issues. Also, you can share parameters across entities in the workflow.
This feature is available in all regions where AWS Glue is available except AWS GovCloud (US-East) and AWS GovCloud (US).
To learn more about this feature, please visit our documentation.