SageMaker HyperPod now supports idle resource sharing for dynamic cluster utilization

Amazon SageMaker HyperPod task governance now supports dynamic resource sharing, allowing teams to borrow unallocated compute capacity in HyperPod clusters beyond their guaranteed quotas. Administrators can also configure borrow limits for specific resource types, such as accelerators, vCPU, or memory, to ensure fair distribution across teams.

Administrators running shared compute clusters for generative AI workloads often face underutilization challenges. When data scientists do not fully consume their allocated quotas, expensive compute instances remain idle. Idle resource sharing solves this by automatically identifying unallocated cluster capacity and making it available for teams to borrow on a best-effort basis. HyperPod task governance monitors your cluster state and automatically recalculates borrowable resources when instances and compute quota policies change, eliminating manual configuration. Eligible instances that are in a ready and schedulable state, including instances with partitioned GPU configurations, contribute to the borrowable pool of unallocated compute capacity. Administrators can also define absolute borrow limits in addition to percentage-based borrow limits of idle compute. This helps administrators maximize compute utilization and maintain fine-grained control over how idle capacity is distributed across teams, while ensuring guaranteed compute quota isolation for each team.

This capability is currently available for Amazon SageMaker HyperPod clusters using the EKS orchestrator across the following AWS Regions: US East (N. Virginia), US East (Ohio), US West (N. California), US West (Oregon), Asia Pacific (Mumbai), Asia Pacific (Singapore), Asia Pacific (Sydney), and Asia Pacific (Tokyo), Asia Pacific (Jakarta), Europe (Frankfurt), Europe (Ireland), Europe (London), Europe (Stockholm), Europe (Spain), and South America (São Paulo).

To learn more, visit SageMaker HyperPod webpage, and HyperPod task governance documentation.