S3 conditional write PutObject request does not work correctly with implicit retries
Describe the bug
One can use conditional writes with either PutObject or CompleteMultipartUpload to effectively put object only if it doesn't exist yet (If-None-Match: *).
This works great in principle. This fails when there is a network issue and AWS SDK implicitly retries the underlying request:
- first request goes through and reaches the server, creating the object
- the response does not make it to the client though
- the client retries the request
- this second request fails with HTTP Status-Code 412: Precondition Failed
Regression Issue
- [ ] Select this option if this issue appears to be a regression.
Expected Behavior
PutObject should remain idempotent also when If-None-Match: * condition is used, as far as retries done by the library are concerned.
in other words, the application layer should be able to use If-None-Match: * write condition without doing a lot additional work on the calling side
Current Behavior
PutObject If-None-Match: * write condition leads to application errors when networking circumstances lead to a request being retried on the calling side.
Reproduction Steps
Any PutObject that sets software.amazon.awssdk.services.s3.model.PutObjectRequest.Builder.ifNoneMatch to * can be used to reproduce the issue, but full reproduction requires injecting network failures.
If this helps, i can try to produce such a test case with shopify/toxiproxy. Let me know if I should spend time on this.
Possible Solution
Tagging put objects creations with something that can identify them on the retried request.
Additional Information/Context
No response
AWS Java SDK version used
2.39.1
JDK version used
25
Operating System and version
MacOS 15.6
@findepi have a couple of questions, just to clarify the use case:
- The ask is to change the behavior and make
PutObjectIf-None-Match: *idempotent in general? - Why getting
412 - Precondition Failedas a retry response is a pain in this case? I understand that getting no exceptions is ideal, but given the response didn't reach the SDK in the first attempt, getting the exception in the second attempt is unfortunate but it is expected
2. getting the exception in the second attempt is unfortunate but it is expected
if you view this from "is it working as implemented?" perspective, then yes, it does as implemented
however, let's think about use-case when someone uses AWS SDK and invokes PutObjectRequest.Builder.ifNoneMatch with * argument.
Is the user expectation to "create a file unless it exists"?
or is the user expectation to "create a file unless it exists, or unless the requests gets implicitly retried by the library"?
At least my expectation would be "create a file unless it exists" (with no strings attached) and I would hope the library can do this for me. If it doesn't, then PutObjectRequest.Builder.ifNoneMatch is very hard to use correctly, which suggests it's used incorrectly in majority of situations.