cloudstorage icon indicating copy to clipboard operation
cloudstorage copied to clipboard

S3 Copy/Move performance optimizations

Open araddon opened this issue 7 years ago • 1 comments

There are Copy/Move that are not currently implemented on S3. These are performance optimizations that when copy/move from s3 -> to s3 don't do network transfer, instead allow s3 to do the copy/move through native s3 api.

https://github.com/lytics/cloudstorage/blob/27319609bc1c8b4108e8cbc652c5dcbcf6747145/store.go#L73-L85

Google implementation https://github.com/lytics/cloudstorage/blob/27319609bc1c8b4108e8cbc652c5dcbcf6747145/google/store.go#L174

araddon avatar Jul 11 '18 11:07 araddon

Not entirely sure this can be pushed down, the copy partially can: https://stackoverflow.com/questions/47334556/aws-sdk-go-s3-move-directory-between-bucket https://stackoverflow.com/questions/28830839/aws-s3-move-an-object-to-a-different-folder

araddon avatar Jul 15 '18 22:07 araddon