athena-federation-python-sdk
athena-federation-python-sdk copied to clipboard
Partition Support
Partitions are not supported, so Athena will not parallelize the query using partitions.
The main reason we wanted to use this SDK was to create Athena tables build from multiple individual file partitions, as in:
https://github.com/quiltdata/quilt-package-metadata-athena/blob/main/04_create_datalake_and_athena_tables.ipynb
In particular, we wanted to use Athena Federation directly rather than having to hack it with symlinks.
This appears possible with Java, though I'm not entirely sure how.
Would you expect that to work? If not, how hard do you think it might be to add full Partition support?