Amazon Elastic MapReduce now supports unlimited steps on AMI 3.1.1+
We have removed the 256 step limit for Amazon Elastic MapReduce (EMR) clusters with AMI 3.1.1 and later. A step is a unit of work you can submit to your cluster. Steps are executed sequentially and can contain one or more Hadoop jobs or instructions.
We have removed the 256 step limit for Amazon Elastic MapReduce (EMR) clusters with AMI 3.1.1 and later. A step is a unit of work you can submit to your cluster. Steps are executed sequentially and can contain one or more Hadoop jobs or instructions.
This change allows you to add an unlimited number of steps over the lifetime of your cluster. Although the number of total steps over the cluster lifetime is now unlimited, you can only have up to 256 active or pending steps at any given point in time. Amazon EMR will only keep a record of your last 1000 steps.
For AMIs 3.1.0 and earlier, there is still a limit of 256 total steps over the lifetime of the cluster.
To learn more about creating and adding steps in Amazon EMR, see here.