GZIP Request and Response Configuration

[sedovalx]

Here is how I create a client:

AthenaClient
    .builder()
    .region(Region.of(environment.aws.region))
    .credentialsProvider(DefaultCredentialsProvider.create())
    .overrideConfiguration {
        it.apiCallTimeout(Duration.ofMillis(environment.athena.executionTimeout.toLong()))
//                    it.gzipEnabled(true)
//                    it.totalExecutionTimeout(Duration.ofMillis(environment.athena.executionTimeout.toLong()))
    }
    .build()

Previously I could use the gzipEnabled property to enable gzip encoding, it is absent now. Can you give me an example of setting it up?

[millems]

Right now, GZIP usage is determined automatically based on the service and HTTP implementation itself. There's no flag to force it on. Is this causing you latency issues?

[sedovalx]

I'm doing queries to Athena and its responses might be relatively big json entities. So yes, it may cause problems but I haven't done any measurements yet. Just migrating from 2.0.0-preview-9 and have noticed that response data from Athena came without compressing. The same for DynamoDb.

[millems]

Migrating this to a feature request to add the option of enabling HTTP-level compressing. This would be on the HTTP client implementation itself.

[shorea]

You MAY be able to workaround this by adding a custom header Accept-Encoding: gzip.

Something like this

GetQueryResults.builder()
   .overrideConfiguration(o -> o.putHeader("Accept-Encoding", "gzip"))
   .build()

[nidhinr007]

software.amazon.awssdk.core.exception.Crc32MismatchException. Please help to fix

[cristianocca]

You MAY be able to workaround this by adding a custom header Accept-Encoding: gzip.

Something like this

GetQueryResults.builder()
   .overrideConfiguration(o -> o.putHeader("Accept-Encoding", "gzip"))
   .build()

This doesn't really work, adds the header but the request is no compressed at all.

[roamariei]

Hi,

@millems Based on PR https://github.com/aws/aws-sdk-java-v2/pull/4371, it appears that compression configuration was introduced to the SDK in 2023. Using this configuration, I was able to successfully compress the payload for a PUT request on DynamoDbClient:

     // Enable GZIP compression for requests above 1024 bytes
     CompressionConfiguration compressionConfig = CompressionConfiguration.builder()
            .requestCompressionEnabled(true)
            .minimumCompressionThresholdInBytes(1024)
            .build();

    RequestCompression requestCompressionTrait = RequestCompression.builder()
            .encodings("gzip")
            .isStreaming(false)
            .build();

    // Apply compression to the client
    ClientOverrideConfiguration overrideConfig = ClientOverrideConfiguration.builder()
            .compressionConfiguration(compressionConfig)
            .putExecutionAttribute(SdkInternalExecutionAttribute.REQUEST_COMPRESSION,
                    requestCompressionTrait)
            .build();
    
    DynamoDbClient dynamoDbClient = DynamoDbClient.builder()
            .credentialsProvider(credentialsProvider())
            .region(Region.of(environment.aws.region))
            .overrideConfiguration(overrideConfig)
            .build();

    PutItemResponse response = dynamoDbClient.putItem(putItemRequest);

Did I understand correctly that this PR resolves the linked GitHub issue?