Amazon Athena adds support for Partition Projection
Amazon Athena has added support for Partition Projection, a new functionality that you can use to speed up query processing of highly partitioned tables and automate partition management.
Amazon Athena has added support for Partition Projection, a new functionality that you can use to speed up query processing of highly partitioned tables and automate partition management.
When processing queries, Athena retrieves metadata information from your metadata store such as AWS Glue Data Catalog or your Hive Metastore before performing partition pruning. In cases when your tables have a large number of partitions, retrieving metadata can be time consuming. To avoid this, you can use partition projection. Partition projection allows you to specify configuration information such as the patterns (for example, YYYY/MM/DD) that are commonly used to form partitions. This gives Athena the information necessary to build partitions without retrieving metadata information from your metadata store. Athena will read the partition values and locations from configuration, rather than from a repository like the AWS Glue Data Catalog. Partition projection reduces the runtime of queries against highly partitioned tables since in-memory operations are often faster than remote operations.
Please see our documentation for detailed information and usage examples.