Uploaded Zip files may not be found. S3 multipart upload may be the cause

[keitarou]

Describe the bug

Uploaded Zip files may not be found. Occurrence of error

fatal error: An error occurred (404) when calling the HeadObject operation: Key "public/test-scenarios/jmeter/XXXXXX.zip" does not exist

To Reproduce

Upload the zip schedule and run the test. Files larger than 5MB are suspicious

Expected behavior

From 3.2.6, amplify/storage uploadData can be used when uploading zip files. https://github.com/kamyarz-aws/distributed-load-testing-on-aws/blob/release/v3.2.6/source/console/src/Components/Create/Create.js#L288

With UploadData being used, we found that S3 multi part upload is called for zip files with a capacity of 5MB or more. ・threshold constant: https://github.com/aws-amplify/amplify-js/blob/1625424a5f04ed6eb2ad9386ffc31d8e25f88e90/packages/storage/src/providers/s3/utils/constants.ts#L16 ・Condition https://github.com/aws-amplify/amplify-js/blob/aws-amplify%406.0.16/packages/storage/src/providers/s3/apis/uploadData/index.ts#L70

I couldn't understand whether the S3 multipart completion handling was implemented correctly. https://docs.aws.amazon.com/ja_jp/AmazonS3/latest/API/API_CompleteMultipartUpload.html#API_CompleteMultipartUpload_RequestBody

The processing of a CompleteMultipartUpload request could take several minutes to finalize. After Amazon S3 begins processing the request, it sends an HTTP response header that specifies a 200 OK response. While processing is in progress, Amazon S3 periodically sends white space characters to keep the connection from timing out. A request could fail after the initial 200 OK response has been sent. This means that a 200 OK response can contain either a success or an error. The error response might be embedded in the 200 OK response. If you call this API operation directly, make sure to design your application to parse the contents of the response and handle it appropriately.

The following response indicates that an error occurred after the HTTP response header was sent. Note that while the HTTP status code is 200 OK, the request actually failed as described in the Error element.

DLT uploadData call only performs try~catch, so the HTTP body message is not checked. https://github.com/kamyarz-aws/distributed-load-testing-on-aws/blob/release/v3.2.6/source/console/src/Components/Create/Create.js#L288

Please complete the following information about the solution:

• [ ] Version: from 3.2.6
• [ ] Region: [e.g. us-east-1]
• [ ] Was the solution modified from the version published on this repository? No
• [ ] If the answer to the previous question was yes, are the changes available on GitHub?
• [ ] Have you checked your service quotas for the services this solution uses?
• [ ] Were there any errors in the CloudWatch Logs?

Screenshots

Additional context

[kamyarz-aws]

@keitarou Does the test fails for you ? Cause I can see your point, but its not clear to me where you are losing the functionality. I ran tests with 35 mb files and the tests were executed fine.

[kamyarz-aws]

Closing the issue given the inactivty and inability to generate the issue on our end.