S3ClientBuilder will not use the system configured https.proxyHost

[kycrow32]

Describe the bug

By default, S3ClientBuilder will not use the system configured https.proxyHost. Instead it uses the system configured http.proxyHost

Expected Behavior

S3ClientBuilder should connect through the proxy specified by https.proxyHost.

Current Behavior

S3ClientBuilder ignores the value in https.proxyHost. If http.proxyHost is configured, it will connect through that proxy even though it is using an https connection.

Reproduction Steps

1. Unzip s3client.zip
2. Build s3client.jar with "gradlew clean build"
3. set up environment variables: AWS_KEY_ID=some aws access key AWS_KEY_SECRET=some aws secret key AWS_REGION=some aws region
4. set up https proxies export _JAVA_OPTIONS="-Dhttps.proxyHost=SOME_PROXY_HOST -Dhttps.proxyPort=SOME_PROXY_PORT"
5. run java -DLOGGING_LEVEL_ORG_APACHE_HC=DEBUG -DSOFTWARE_AMAZON_AWSSDK=DEBUG -jar s3client.jar
6. if the proxy really is necessary, see a failure! Exception in thread "main" software.amazon.awssdk.core.exception.SdkClientException: Unable to execute HTTP request: Connection reset at software.amazon.awssdk.core.exception.SdkClientException$BuilderImpl.build(SdkClientException.java:111) at software.am s3client.zip azon.awssdk.core.exception.SdkClientException.create(SdkClientException.java:47)
7. in the output logs, see that NO proxy was picked up. You will NOT see "ApacheHttpClient - Configuring Proxy. Proxy Host: SOME_PROXY_HOST"
8. set up http proxies export _JAVA_OPTIONS="-Dhttps.proxyHost=SOME_PROXY_HOST -Dhttps.proxyPort=SOME_PROXY_PORT -Dhttp.proxyHost=SOME_PROXY_HOST -Dhttp.proxyPort=SOME_PROXY_PORT"
9. run java -DLOGGING_LEVEL_ORG_APACHE_HC=DEBUG -DSOFTWARE_AMAZON_AWSSDK=DEBUG -jar s3client.jar
10. see a successful list of your s3 buckets!
11. in the output logs, see that a proxy was picked up. You will see "ApacheHttpClient - Configuring Proxy. Proxy Host: SOME_PROXY_HOST"

Possible Solution

No response

Additional Information/Context

It looks like this happens becuase software.amazon.awssdk.http.apache.ProxyConfiguration won't set a host if no endpoint was provided, and an endpoint is not provided by default. So by default, the S3Client (and I assume other aws clients) will connect using https (good), but use the http.proxyHost system setting, instead of the https.proxyHost system setting (bad).

I tried calling s3ClientBuilder.endpointOverride("https://HOST:PORT"), but S3Client continued to fail to connect. And endpointOverride looks like it is expecting an S3 URL anyway.

AWS Java SDK version used

2.20.30

JDK version used

OpenJDK 17.0.6

Operating System and version

Oracle Linux 8.7