rustfs
Add support for `If-None-Match` header (ETag-based conditional GET)
Is your feature request related to a problem? Please describe.
Currently, RustFS does not support the If-None-Match HTTP header for conditional GET requests. This prevents clients from efficiently checking if an object has changed, resulting in unnecessary downloads and wasted bandwidth.
Describe the solution you'd like
Add support for the If-None-Match header so that RustFS can respond with 304 Not Modified when the object's ETag matches. This would improve caching efficiency and compatibility with S3 clients that rely on ETag validation.
Describe alternatives you've considered There is no practical alternative: conditional GET based on ETag requires server-side support.
Additional context https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObject.html#API_GetObject_RequestSyntax
s3s has parsed If-None-Match header into the input struct.
This feature needs implementation on rustfs side.
Hey @sunfkny ,
This feature is now implemented. Could you share your test plan or the cases you'll be using for verification?
https://github.com/rustfs/rustfs/pull/885
@loverustfs
There is a bug in ETag comparison: info.etag is unquoted (Some("440747666b49289943af5176aab06a47")) while if_none_match is wrapped in quotes (Some("\"440747666b49289943af5176aab06a47\"")). This causes requests that should return 304 Not Modified to return 200 OK.
https://github.com/rustfs/rustfs/blob/494fe7d71e3218c43a4753153ce823de4ba87a4f/rustfs/src/storage/ecfs.rs#L1673-L1677
Test plan
-
Create a public bucket (e.g.,
test-bucket) and upload a file (e.g.,README.md) to it. -
Save the ETag of the file.
curl --etag-save etag.txt -LsSf http://localhost:9000/test-bucket/README.md -o /dev/null -
Get object again with the ETag. Check http status code, it should be 304.
curl --etag-compare etag.txt -LsSf http://localhost:9000/test-bucket/README.md -w 'status code: %{http_code}' -v -o /dev/null
Milvus startup exception: panic: CheckIfConditionWriteSupport failed: PutObjectIfNoneMatch not supported or failed. BucketName: milvus, ObjectKey: files/wp/conditional_write_test_object, Error: The specified key does not exist.
it seems "If-None-Match" returns PreconditionFailed, causing milvus to fail to start, in put_object and complete_multipart_upload。
Doesn't it need to be modified in complete_multipart_upload?
@shiroleeee
Doesn't it need to be modified in complete_multipart_upload?
Hey @halfpreschooler ,
Thank you very much.
https://github.com/rustfs/rustfs/pull/1065
Fixed.
Hey @halfpreschooler @sunfkny ,
This bug has been fixed in the latest release. I'm closing this issue for now.
https://github.com/rustfs/rustfs/releases/tag/1.0.0-alpha.73
If the problem persists, please leave a comment and we'll reopen it.
Hey @halfpreschooler @sunfkny ,
This bug has been fixed in the latest release. I'm closing this issue for now.
https://github.com/rustfs/rustfs/releases/tag/1.0.0-alpha.73
If the problem persists, please leave a comment and we'll reopen it.
@loverustfs
Thanks for the update. This issue https://github.com/rustfs/rustfs/issues/791#issuecomment-3550337057 has not been resolved yet.
The underlying fix https://github.com/s3s-project/s3s/pull/403 was made in s3s and has already been merged and released in v0.12.0-rc.5. ETag is now strongly typed, and once RustFS upgrades to this version, the type system will prevent incorrect ETag comparisons and ensure correct conditional request behavior.
Edit: FYI, in PR #1008, get_object was refactored, but this effectively removed the precondition check logic. The intention was to move the checks to precondition_check, but get_object does not actually call get_object_info, which includes the precondition checks.
Hey @sunfkny ,
Thank you for using RustFS.
This bug has been fixed. Please upgrade to the latest version, alpha.75: https://github.com/rustfs/rustfs/releases/tag/1.0.0-alpha.75
Please verify that this resolves the issue. If the problem persists, please leave a comment below and we will reopen this issue.