aws-sdk-pandas icon indicating copy to clipboard operation
aws-sdk-pandas copied to clipboard

Enable Ray distribution on read_parquet method

Open jaidisido opened this issue 3 years ago • 0 comments

Is your feature request related to a problem? Please describe. Ensure wr.s3.read_parquet runs at scale leveraging Ray

Additional context This is part of the Wrangler @ Scale release

Links

  • https://docs.ray.io/en/latest/data/key-concepts.html
  • https://docs.ray.io/en/latest/data/package-ref.html#creating-datasets
  • https://docs.ray.io/en/latest/data/package-ref.html#custom-datasource-api
  • https://docs.ray.io/en/latest/data/package-ref.html#built-in-datasources

P.S. Please do not attach files as it's considered a security risk. Add code snippets directly in the message body as much as possible.

jaidisido avatar Aug 02 '22 09:08 jaidisido