cloud-storage-rs
cloud-storage-rs copied to clipboard
Add range option for download requests
This commit adds the option to pass range headers into the download requests so that one may satisfy the datafusion ObjectReader interface with this library that requires a chunked reader option.
I'm working on making a GCS version of the S3 datafusion https://github.com/datafusion-contrib/datafusion-objectstore-s3 adapter and using cloud-storage-rs for that.
Mostly a weekend warrior with rust so let me know if this would be sane or if I could do something to improve this!
For reference the gcs adapter I'm working on is here https://github.com/ajroetker/datafusion-objectstore-gcs
@ThouCheese not sure if I need to ping you or anything for this :)