strato icon indicating copy to clipboard operation
strato copied to clipboard

Issue While doing Multi-part Upload on GCP (google cloud) [Just Verify]

Open click2cloud-alpha opened this issue 5 years ago • 8 comments

Test Scenario: Performing migration of large data from AWS to GCP File Size : 1.0 GB File name : CentOs6.vhd Error log ss

Reason : GCP doesn’t provide Multipart Upload. Blocking : we can achieve it with GCP parallel upload but OpenSDS multi-cloud common controller architecture supports multi-part API not composite and parallel upload proposed by GCP . Limitation : GCP has limitation for multipart upload.

Reference link: https://stackoverflow.com/questions/27830432/google-cloud-storage-support-of-s3-multipart-upload https://cloud.google.com/storage/docs/composite-objects

suggested Solution : if we need to resolve this , we would need your suggestion like how we can manage this parallel and composite upload of GCP with OpenSDS Multi-cloud Controller architechture.

click2cloud-alpha avatar Apr 09 '19 06:04 click2cloud-alpha

It seems gcp do support multipart upload, and our multi-cloud architecture is compatible with it, but we need more time to fix it. See: QQ图片20190419152616

sfzeng avatar Apr 19 '19 07:04 sfzeng

@nguptaopensds / @sfzeng can we consider this for capri ? Please comment.

skdwriting avatar Jun 11 '19 09:06 skdwriting

https://cloud.google.com/storage/docs/json_api/v1/how-tos/multipart-upload

kumarashit avatar Jun 25 '19 06:06 kumarashit

https://www.zenko.io/blog/four-differences-google-amazon-s3-api/

kumarashit avatar Jun 25 '19 06:06 kumarashit

is this issue closed> ? @PravinRanjan10

shelar1423 avatar Mar 30 '22 14:03 shelar1423

@shelar1423 not yet. please take it and verify

PravinRanjan10 avatar Apr 02 '22 03:04 PravinRanjan10

WIll verify

merothh avatar Apr 25 '22 15:04 merothh

WIll verify

Uploading a 512M file stills fails.

image

merothh avatar Apr 26 '22 18:04 merothh