zio-connect icon indicating copy to clipboard operation
zio-connect copied to clipboard

S3 connector's putObject is wrong

Open vigoo opened this issue 1 year ago • 2 comments

It seems to repeatedly call putObject for each chunk, which just overwrites the same object every time. It should use the multipart upload API: https://docs.aws.amazon.com/AmazonS3/latest/userguide/mpuoverview.html

vigoo avatar May 20 '23 16:05 vigoo

Is there a workaround for this?

bradovitt avatar May 02 '24 21:05 bradovitt

One workaround is to rechunk your ZStream to have 1 underlying chunk.

bradovitt avatar May 02 '24 21:05 bradovitt