cloud-storage-rs
cloud-storage-rs copied to clipboard
download a stream of bytes, not single byte
Hi!
I noticed that fact that your download_streaming APIs return an impl Stream<Item = crate::Result<u8>>
which is very inefficient. create::Result<u8>
has size 72 on my machine (64bit arm), so for every byte copied, we need to pass 72 bytes around. But even if it was impl Stream<Item = u8>
, it would still be inefficient. Such streams always work on slices of bytes (e.g. see the Read or AsyncRead traits) or here, it's the easiest and most idiomatic to just return a stream of Bytes
.
So I deprecated download_streamed
and added download_bytes_stream
.
Hope you like it :)
Cheers, Kosta
Actually, I noticed that I not only need the stream of bytes but also at least the content-length for my use case (which is provided in a header). I will update my MR accordingly.
Nevermind my last comment, please review this and merge if you feel this is appropriate.
Hi! This PR is open since October and is +96 −32 across 3 files and I think it's pretty straightforward. I would appreciate you taking the time to review it, but of course I understand if you cannot make the time to do it. If you think this PR does not improve your codebase, please close it. Thank you!