s5cmd
s5cmd copied to clipboard
Common mutipart signature auth error can cause multipart uploads to fail
Common multipart uploads fail with an error like:
ERROR "cp **** s3://****": MultipartUpload: upload multipart failed upload id: dBmwIX7UhK9IotwKi1j1bM.sInQ7jUngrmjZ3GlmNz1GfLmYZ_4b94ReAgZ4OOMm0wQUfm2DQshNxlCyTBFuzlOfMWKnMLIhEYVB7p5iw39vMcNJn375vBDhWH069xCaa24d8SyktuEyNR5jIt.EOGp9fiucRRtXZ9l80.UnARw- caused by: SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your key and signing method. status code: 403, request id: H871REM5J5C1BNN0, host id: FE1FVxCXCymin8yLl1DfjNAyx/yWqS9MwSQ0b7Tcum5xEeoMo/nbpZOjqgEtdnsx2XrDFqrR6/0=
Typically re-uploading the failed object succeeds. It may be good to add retries for this type of error (even though it is a typically non-retry auth error)
I should add this was observed uploading to an s3 bucket with transfer acceleration enabled using a command like this:
s5cmd --endpoint-url https://****.s3-accelerate.amazonaws.com --numworkers 512 cp --destination-region eu-central-1 --concurrency 512 **** s3://*****
I'm having the same issue with R2