azure-sdk-for-java icon indicating copy to clipboard operation
azure-sdk-for-java copied to clipboard

[QUERY] download blob in parallel using openinputstream

Open ihudedi opened this issue 4 years ago • 4 comments

Hi, I would like to know how can I download blob using stream in paralel.i found via downloadtofile it's exist but i would like to know if it can be using openinputstream or downloadstream. Thanks, Itay.

ihudedi avatar Oct 08 '21 10:10 ihudedi

@ihudedi The only parallelized download we offer right now is downloading to a file as it seems to be the only meaningful place to add that behavior. If a download is big enough to be meaningfully affected by parallelizing requests, then it tends to be large enough that holding it all in memory is a bad idea.

Do you have a use case for this?

rickle-msft avatar Oct 08 '21 17:10 rickle-msft

Hi @rickle-msft We transfer files from azure to any destination.the destination is outputstream.when downloadtofile occur do you save all parts in the file system and after that you merge to one file? Thanks, Itay

ihudedi avatar Oct 09 '21 19:10 ihudedi

@ihudedi When downloading to a file, we open an AsynchronousFileChannel, which supports concurrent writes by specifying the desired offset along with each block of data. So we actually write into the file in "parallel" (I'm sure its serialized at some point since there's probably only one pointer to the disk, but as far as the jvm can tell it's parallel).

Typically an outputstream does not support seeking operations, right?

rickle-msft avatar Oct 11 '21 18:10 rickle-msft

Hi @ihudedi ,

Just following up on this issue. Has this issue been resolved? If so, we can go ahead and close this. If not, can you please let us know what blockers you are running into, or any other questions you may have?

Thank you!

ibrahimrabab avatar Sep 23 '22 21:09 ibrahimrabab

Hi @ihudedi, we're sending this friendly reminder because we haven't heard back from you in 7 days. We need more information about this issue to help address it. Please be sure to give us your input. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you!

github-actions[bot] avatar Apr 13 '23 09:04 github-actions[bot]