s3-utils icon indicating copy to clipboard operation
s3-utils copied to clipboard

concat should support parallel writing

Open JoshWillik opened this issue 3 years ago • 0 comments

The reason I was interested in this tool was to merge ~2TB (3gb chunks) of data without downloading it to a middleman server first. Doing this in serial will take too long. I'm not great with rust but I I ended up creating a copycat project in javascript to do this https://gitlab.com/joshwillik/s3-concat (in case you want to read my implementation)

JoshWillik avatar Sep 27 '22 19:09 JoshWillik