paws icon indicating copy to clipboard operation
paws copied to clipboard

Published 20 hours ago • verified publisher icon paws-r

`s3` download directly to a file?

Open dereckmezquita opened this issue 1 year ago • 3 comments

I might have missed something but after consulting the examples on S3 all I see are methods for requesting something from S3 and storing the raw response in a variable.

I would like to download things directly to files. I am dealing with very large files and it is not possible for me to store all of this data in memory. I need to have the files downloaded and then I can work on them.

Is there a way to do this?

dereckmezquita avatar Aug 12 '22 17:08 dereckmezquita

Currently it going to memory then to file. We do have a PR (https://github.com/paws-r/paws/pull/458) that will output directly to disk however I believe there needs to be a few tweaks to it to get it in probably :)

DyfanJones avatar Aug 12 '22 18:08 DyfanJones

Okay then thank you very much anywho, nice package. R is the main workhorse for thousands and your work keeps at the edge :)

dereckmezquita avatar Aug 12 '22 22:08 dereckmezquita

Hi @dereckdemezquita PR (#458) has been merged, and will be in the next paws release

DyfanJones avatar Aug 16 '22 16:08 DyfanJones

Closing this ticket due to paws.storage 0.2.0 has been released to the cran

DyfanJones avatar Feb 08 '23 14:02 DyfanJones