AWS DataSync now supports manifests for transferring a specific set of files

AWS DataSync introduces manifests, a new feature that enables you to provide a definitive list of source files or objects to be transferred by your DataSync tasks. Using manifests, you can decrease your task execution times by specifying only the files or objects that need to be processed by your task.

AWS DataSync is an online data movement service. DataSync automates and accelerates copying data between AWS Storage services, on-premises storage, edge locations, or other clouds. When a DataSync task executes, it scans and compares source and destination locations to determine which files or objects should be transferred. For large file systems or object stores, the scan and compare operation can significantly increase the overall duration of the task execution. If you have well-known datasets, such as those moved as part of an automated workflow, you can use manifests to avoid scanning your entire file or object storage systems to determine changes. Using a manifest file, you can specify millions of source files or objects to be transferred, and DataSync will only scan and compare the files listed in the manifest. You can also use manifests to copy specific versions of objects from your Amazon S3 bucket.

You can use manifests in all AWS Regions where AWS DataSync is available, except the China (Beijing) and the China (Ningxia) Regions. To get started, visit the AWS DataSync console. To learn more, view the AWS DataSync documentation.