Not able to use PutObject after update

[torjo255]

Describe the bug

After switching from 1.8.186 to 1.10.49 my PutObject request started to fail due to “The Conten-MD5 you specified did not match what we received” However, I did not specify an MD5 and when I look into the code of the SDK (PutObjectRequest::GetChecksumAlgroithmNam) it seems that MD5 is selected by default if no Algorithm was selected. Since the stream I have provided with SetBody is still being written to when the PutObject is started (we stream data from the network) the SDK cannot calculate a correct MD5.

Expected Behavior

MD5 should not be set automatically when no algorithm was provided.

Current Behavior

MD5 is set and since we stream data we cannot calculate it before the streaming is completed.

Reproduction Steps

Use PutObject with a stream that is not fully written to when the PutObject start.

Possible Solution

Allow the user to use ChecksumAlgorithm::NOT_SET

Additional Information/Context

No response

AWS CPP SDK version used

1.10.49

Compiler and Version used

8.3.1

Operating System and version

Centos7

[jmklix]

Can you provide a minimal sample that shows PutObject not working?

[torjo255]

Sure!

This will work on 1.8.186 but not on 1.10.49.

Aws::SDKOptions options;
Aws::InitAPI(options);
Aws::Client::ClientConfiguration clientConfig;
client = std::make_shared<Aws::S3::S3Client>(clientConfig);

auto data = std::make_shared<Aws::StringStream>(
    "put", std::stringstream::in | std::stringstream::out | std::stringstream::binary);

data->write("foo", 3);
auto uploadRequest = PutObjectRequest();
uploadRequest.SetBucket(bucket);
uploadRequest.SetKey(key);
uploadRequest.SetContentLength(6);
uploadRequest.SetBody(data);
auto outcomeCallable = client->PutObjectCallable(uploadRequest);

// Wait for put object to start until finishing writing to stream (simulate streaming traffic
// from network)
std::this_thread::sleep_for(std::chrono::milliseconds(500));
data->write("bar", 3);
auto result = outcomeCallable.get();

if (result.IsSuccess()) {
    std::cerr << "Success!";
}
else {
    std::stringstream errorMsg;
    errorMsg << result.GetError().GetExceptionName() << " " << result.GetError().GetMessage();
    std::cerr << errorMsg.str();
}

[torjo255]

I forgot to mention that the same problem occurs for UploadPart

[lucaspeng12138]

I think this issue is related to what I found : https://github.com/aws/aws-sdk-cpp/discussions/2433 The request always calculate MD5 and ignore what I set in request.

[jmklix]

Last year we made some changes and bug fixes when calculating checksums. So the above error shouldn't be a problem anymore. Your sample code isn't structured correctly to match our suggested use pattern. All sdk calls should be within brackets as show here. We also have a full pub_object sample here that shows you how to use this sdk. Can you please try using a latest version of this sdk and let me know if you run into any problems?

[github-actions[bot]]

Greetings! It looks like this issue hasn’t been active in longer than a week. We encourage you to check if this is still an issue in the latest release. Because it has been longer than a week since the last update on this, and in the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please feel free to provide a comment or add an upvote to prevent automatic closure, or if the issue is already closed, please feel free to open a new one.