cloud-storage-rs icon indicating copy to clipboard operation
cloud-storage-rs copied to clipboard

download a stream of bytes, not single byte

Open kosta opened this issue 2 years ago • 3 comments

Hi!

I noticed that fact that your download_streaming APIs return an impl Stream<Item = crate::Result<u8>> which is very inefficient. create::Result<u8> has size 72 on my machine (64bit arm), so for every byte copied, we need to pass 72 bytes around. But even if it was impl Stream<Item = u8>, it would still be inefficient. Such streams always work on slices of bytes (e.g. see the Read or AsyncRead traits) or here, it's the easiest and most idiomatic to just return a stream of Bytes.

So I deprecated download_streamed and added download_bytes_stream.

Hope you like it :)

Cheers, Kosta

kosta avatar Oct 07 '22 11:10 kosta

Actually, I noticed that I not only need the stream of bytes but also at least the content-length for my use case (which is provided in a header). I will update my MR accordingly.

kosta avatar Oct 21 '22 07:10 kosta

Nevermind my last comment, please review this and merge if you feel this is appropriate.

kosta avatar Jan 02 '23 15:01 kosta

Hi! This PR is open since October and is +96 −32 across 3 files and I think it's pretty straightforward. I would appreciate you taking the time to review it, but of course I understand if you cannot make the time to do it. If you think this PR does not improve your codebase, please close it. Thank you!

kosta avatar Feb 02 '23 08:02 kosta