AWS Batch Adds Support for Large-Scale Job Submissions with Array Jobs
AWS Batch now supports the submission of arrays of jobs with a single API call. With ArrayJobs, users can easily submit, with a single SubmitJob call, up to 10,000 copies of a job. Users can also express dependencies on Array Jobs, including 'N-to-N' dependencies between related elements across multiple Array Jobs. Array Jobs simplify the submission and management of large-scale Monte Carlo simulations, parametric sweeps, and jobs which need to perform the same operation on thousands of input files or objects. AWS Batch dynamically provisions the optimal quantity and type of compute resources (e.g., CPU or memory optimized instances) based on the volume and specific resource requirements of the batch jobs submitted. With AWS Batch, there is no need to install and manage batch computing software or server clusters.
AWS Batch now supports the submission of arrays of jobs with a single API call. With ArrayJobs, users can easily submit, with a single SubmitJob call, up to 10,000 copies of a job. Users can also express dependencies on Array Jobs, including 'N-to-N' dependencies between related elements across multiple Array Jobs. Array Jobs simplify the submission and management of large-scale Monte Carlo simulations, parametric sweeps, and jobs which need to perform the same operation on thousands of input files or objects. AWS Batch dynamically provisions the optimal quantity and type of compute resources (e.g., CPU or memory optimized instances) based on the volume and specific resource requirements of the batch jobs submitted. With AWS Batch, there is no need to install and manage batch computing software or server clusters.
Array Jobs are supported in all regions where AWS Batch is currently available:
- US East (N. Virginia) us-east-1
- US East (Ohio) us-east-2
- US West (Oregon) us-west-2
- EU (Ireland) eu-west-1
- EU (London) eu-west-2
- EU (Frankfurt) eu-central-1
- Asia Pacific (Tokyo) ap-northeast-1
- Asia Pacific (Singapore) ap-southeast-1
- Asia Pacific (Sydney) ap-southeast-2
For more information on AWS Batch: