cloud-storage-rs icon indicating copy to clipboard operation
cloud-storage-rs copied to clipboard

Add range option for download requests

Open ajroetker opened this issue 2 years ago • 2 comments

This commit adds the option to pass range headers into the download requests so that one may satisfy the datafusion ObjectReader interface with this library that requires a chunked reader option.

I'm working on making a GCS version of the S3 datafusion https://github.com/datafusion-contrib/datafusion-objectstore-s3 adapter and using cloud-storage-rs for that.

Mostly a weekend warrior with rust so let me know if this would be sane or if I could do something to improve this!

ajroetker avatar Sep 20 '22 19:09 ajroetker

For reference the gcs adapter I'm working on is here https://github.com/ajroetker/datafusion-objectstore-gcs

ajroetker avatar Sep 20 '22 19:09 ajroetker

@ThouCheese not sure if I need to ping you or anything for this :)

ajroetker avatar Sep 22 '22 16:09 ajroetker