AmazonWebServicesClientProxy.logRequestMetadataV2 causes a premature evaluation of the response object

[osdrv]

Summary: AmazonWebServicesClientProxy.logRequestMetadataV2 causes lazily-evaluated SDK response objects (streams, iterables) to be evaluated immediately. A response resolve would immediately engage the SDK client and execute the service API calls. Whenever a consumer of this response object would access the data again, it would have to be re-resolved and the same API calls would be executed once again. The issue issue exists in the latest lib version.

Example: using injectCredentialsAndInvokeIterableV2 paginated operation causes the SDK to perform 2x more API requests.

Details: AmazonWebServicesClientProxy exposes multiple handles to interact with the SDK client. A response object could be either immediately (plain AwsResponse object) or lazily (CompletableFuture<ResponseT>, SdkIterable<ResponseT>, ResponseInputStream<ResponseT>) evaluated. A private logging routine called logRequestMetadataV2 causes lazily evaluated response objects to be evaluated immediately for the sake of logging. The resolve would cause a full range of the service API calls to be executed. By default, the SDK would not perform a deep response cache, hence a secondary access to the response data would once again hook up the SDK client, which would perform the same set of API calls.

    public <RequestT extends AwsRequest, ResultT extends AwsResponse, IterableT extends SdkIterable<ResultT>>
        IterableT
        injectCredentialsAndInvokeIterableV2(final RequestT request, final Function<RequestT, IterableT> requestFunction) {

        AwsRequestOverrideConfiguration overrideConfiguration = AwsRequestOverrideConfiguration.builder()
            .credentialsProvider(v2CredentialsProvider).build();

        @SuppressWarnings("unchecked")
        RequestT wrappedRequest = (RequestT) request.toBuilder().overrideConfiguration(overrideConfiguration).build();

        try {
            IterableT response = requestFunction.apply(wrappedRequest);
            response.forEach(r -> logRequestMetadataV2(request, r)); // <- this invocation would resolve the response object immediately
            return response; // <- the response object is returned to the invoker. It would be re-resolved upon a data access.
        } catch (final Throwable e) {
            loggerProxy.log(String.format("Failed to execute remote function: {%s}", e.getMessage()));
            throw e;
        }
    }

Possible Mitigation: logRequestMetadataV2 (and any other kind of non-lazy logging) should be avoided on all non-immediately resolved result types in the following routines:

• injectCredentialsAndInvokeV2Async
• injectCredentialsAndInvokeIterableV2
• injectCredentialsAndInvokeV2InputStream
• injectCredentialsAndInvokeV2Bytes