S3 User defined metadata key is case-sensitive

[sis-yoshiday]

Type: Bug

Component: "S3"

Describe the bug User defined metadata key is case-sensitive.

Below document say user-defined metadata key is stored in lower-case.

https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingMetadata.html#object-metadata User-defined metadata is a set of key-value pairs. Amazon S3 stores user-defined metadata keys in lowercase.

Sample

s3Template.upload(
        "bucket",
        "key",
        new ByteArrayInputStream(new byte[] { 0 }),
        ObjectMetadata.builder()
            .metadata("myMetadata", "foo")
            .build());

// ...

S3Client s3Client;

Map<String, String> metadata =
        s3Client.getObject(GetObjectRequest.builder().bucket("bucket").key("key").build())
            .response()
            .metadata();

metadata.get("myMetadata");// null
metadata.get("mymetadata");// "foo"

Can we have metadata as some case-insensitive Map ?

metadata = new TreeMap<String, String>(String.CASE_INSENSITIVE_ORDER);

aws-java-sdk v1

https://github.com/aws/aws-sdk-java/blob/56d8e9fa02a3bd7562e4588f4fbfd2dda545bd12/aws-java-sdk-s3/src/main/java/com/amazonaws/services/s3/model/ObjectMetadata.java#L53

/**
  * Custom user metadata, represented in responses with the x-amz-meta-
  * header prefix
  */
private Map<String, String> userMetadata = 
  new TreeMap<String, String>(String.CASE_INSENSITIVE_ORDER);

In sdk v1, user-defined-metadata key seems to be case-insensitive.

aws-java-sdk v2 (2.17.225)

response metadata seems to be copied by software.amazon.awssdk.services.s3.model.MetadataCopier#copy and finally returns as immutable map and not case-insensitive.

[maciejwalkowiak]

Regarding first piece of code:

s3Template.upload(
        "bucket",
        "key",
        new ByteArrayInputStream(new byte[] { 0 }),
        ObjectMetadata.builder()
            .metadata("myMetadata", "foo")
            .metadata("mymetadata", "boo")
            .build());

we could change ObjectMetadata class to store in such case only mymetadata property with boo value.

Regarding getting object with S3Client - perhaps it's a good candidate for an issue in AWS SDK itself?

[sis-yoshiday]

@maciejwalkowiak Right, it may aws sdk's issue and need to report as aws-java-sdk issue.

Apart from that, it would be nice that we can access user-defined-metadata from S3Resource.

[maciejwalkowiak]

@sis-yoshiday would you like to contribute a PR?

[sis-yoshiday]

@maciejwalkowiak Sure, I saparate issue.

https://github.com/awspring/spring-cloud-aws/issues/449

[maciejwalkowiak]

Closing as this is an SDK issue rather than Spring Cloud AWS