aws-sdk-pandas
aws-sdk-pandas copied to clipboard
Enable Ray distribution on read_parquet method
Is your feature request related to a problem? Please describe.
Ensure wr.s3.read_parquet runs at scale leveraging Ray
Additional context This is part of the Wrangler @ Scale release
Links
- https://docs.ray.io/en/latest/data/key-concepts.html
- https://docs.ray.io/en/latest/data/package-ref.html#creating-datasets
- https://docs.ray.io/en/latest/data/package-ref.html#custom-datasource-api
- https://docs.ray.io/en/latest/data/package-ref.html#built-in-datasources
P.S. Please do not attach files as it's considered a security risk. Add code snippets directly in the message body as much as possible.