aws-sdk-java-v2 S3CrtAsyncClient does not work with AnonymousCredentialsProvider

Describe the bug

When the S3CrtAsyncClientBuilder is configured only with AnonymousCredentialsProvider it generates a NullPointerException.

Regression Issue

[ ] Select this option if this issue appears to be a regression.

Expected Behavior

CrtAsyncClient should work with an open bucket when only AnonymousCredentials are set

Current Behavior

The problem is in the CrtCredentialsProviderAdapter

        this.crtCredentials = new DelegateCredentialsProvider.DelegateCredentialsProviderBuilder()
            .withHandler(() -> {

                if (credentialsProvider instanceof AnonymousCredentialsProvider) {
                    return Credentials.createAnonymousCredentials();
                }

                AwsCredentialsIdentity sdkCredentials =
                    CompletableFutureUtils.joinLikeSync(credentialsProvider.resolveIdentity());
                // next will generate the NPE - 
                // if the credentials are null it should just return new Credentials() - this worked when I ran my program in debugger
                byte[] accessKey = sdkCredentials.accessKeyId().getBytes(StandardCharsets.UTF_8);
                byte[] secreteKey = sdkCredentials.secretAccessKey().getBytes(StandardCharsets.UTF_8);

                byte[] sessionTokens = null;
                if (sdkCredentials instanceof AwsSessionCredentialsIdentity) {
                    sessionTokens =
                        ((AwsSessionCredentialsIdentity) sdkCredentials).sessionToken().getBytes(StandardCharsets.UTF_8);
                }
                return new Credentials(accessKey, secreteKey, sessionTokens);

            }).build();

Reproduction Steps

create a crt async client then run a list or getobject request against an open bucket that does not require any credentials for reading. { S3CrtAsyncClientBuilder asyncS3ClientBuilder = S3AsyncClient.crtBuilder(); asyncS3ClientBuilder.credentialsProvider(AnonymousCredentialsProvider.create()); S3AsyncClient asyncClient = asyncS3ClientBuilder.build();

    ListObjectsV2Request listRequest = ListObjectsV2Request.builder()
            .bucket(s3Adapter.getBucket())
            .prefix(prefix)
            .delimiter("/")
            .build();

    ListObjectsV2Publisher listObjectsV2Publisher = s3Adapter.getAsyncS3Client().listObjectsV2Paginator(listRequest);
    ......

}

Possible Solution

Change the internal.CrtCredentialsProvider's constructor

    public CrtCredentialsProviderAdapter(IdentityProvider<? extends AwsCredentialsIdentity> credentialsProvider) {
        this.credentialsProvider = credentialsProvider;
        this.crtCredentials = new DelegateCredentialsProvider.DelegateCredentialsProviderBuilder()
            .withHandler(() -> {

                if (credentialsProvider instanceof AnonymousCredentialsProvider) {
                    return Credentials.createAnonymousCredentials();
                }

                AwsCredentialsIdentity sdkCredentials =
                    CompletableFutureUtils.joinLikeSync(credentialsProvider.resolveIdentity());
                // I don't know if this is the best but when I changed the values in my debugger to avoid the NPE this worked.
                if (sdkCredentials.accessKeyId() == null || sdkCredentials.secretAccessKey() == null) return new Credentials()

                byte[] accessKey = sdkCredentials.accessKeyId().getBytes(StandardCharsets.UTF_8);
                byte[] secreteKey = sdkCredentials.secretAccessKey().getBytes(StandardCharsets.UTF_8);

                byte[] sessionTokens = null;
                if (sdkCredentials instanceof AwsSessionCredentialsIdentity) {
                    sessionTokens =
                        ((AwsSessionCredentialsIdentity) sdkCredentials).sessionToken().getBytes(StandardCharsets.UTF_8);
                }
                return new Credentials(accessKey, secreteKey, sessionTokens);

            }).build();
    }

Additional Information/Context

No response

AWS Java SDK version used

2.30.1

JDK version used

"1.8.0_422" - OpenJDK 64-Bit Server VM (Zulu 8.80.0.17-CA-macos-aarch64) (build 25.422-b05, mixed mode)

Operating System and version

Mac OS Sequoia 15.2

Jan 17 '25 20:01 cgoina

@cgoina can you share the full stacktrace of the NPE?

Jan 17 '25 22:01 debora-ito

This stack I got with jdk 23 but it fails at the same spot with jdk 8 as I specified in the ticket

2025-01-17_18:01:17.454 [main] DEBUG software.amazon.awssdk.request - Sending Request: DefaultSdkHttpFullRequest(httpMethod=GET, protocol=https, host=aind-msma-morphology-data.s3.us-west-2.amazonaws.com, encodedPath=, headers=[amz-sdk-invocation-id, User-Agent], queryParameters=[list-type, delimiter, prefix])
Exception in thread "AwsEventLoop 4" java.lang.NullPointerException: Cannot invoke "String.getBytes(java.nio.charset.Charset)" because the return value of "software.amazon.awssdk.identity.spi.AwsCredentialsIdentity.accessKeyId()" is null
	at software.amazon.awssdk.services.s3.internal.crt.CrtCredentialsProviderAdapter.lambda$new$0(CrtCredentialsProviderAdapter.java:51)
2025-01-17_18:01:17.464 [sdk-async-response-0-0] DEBUG s.a.a.r.i.DefaultStandardRetryStrategy - Request attempt 1 encountered non-retryable failure
software.amazon.awssdk.core.exception.SdkClientException: Failed to send the request: A callback has reported failure.
	at software.amazon.awssdk.core.exception.SdkClientException$BuilderImpl.build(SdkClientException.java:111)
	at software.amazon.awssdk.core.exception.SdkClientException.create(SdkClientException.java:47)
	at software.amazon.awssdk.services.s3.internal.crt.S3CrtResponseHandlerAdapter.handleIoError(S3CrtResponseHandlerAdapter.java:192)
	at software.amazon.awssdk.services.s3.internal.crt.S3CrtResponseHandlerAdapter.handleError(S3CrtResponseHandlerAdapter.java:184)
	at software.amazon.awssdk.services.s3.internal.crt.S3CrtResponseHandlerAdapter.onFinished(S3CrtResponseHandlerAdapter.java:154)
	at software.amazon.awssdk.crt.s3.S3MetaRequestResponseHandlerNativeAdapter.onFinished(S3MetaRequestResponseHandlerNativeAdapter.java:25)
2025-01-17_18:01:17.467 [main] DEBUG o.j.j.s.i.AbstractS3StorageService - List content segmentation/exaSPIM_653159_zarr with org.janelia.jacsstorage.service.ContentAccessParams@4ed9f7b1[filterType=<null>,selectedEntries=[],entryNamePattern=<null>,maxDepth=1,filterTypeSpecificParams={}] - 0.074 secs

java.util.concurrent.CompletionException: software.amazon.awssdk.core.exception.SdkClientException: Failed to send the request: A callback has reported failure.

	at software.amazon.awssdk.utils.CompletableFutureUtils.errorAsCompletionException(CompletableFutureUtils.java:64)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncExecutionFailureExceptionReportingStage.lambda$execute$0(AsyncExecutionFailureExceptionReportingStage.java:51)
	at java.base/java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:978)
	at java.base/java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:955)
	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:554)
	at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2238)
	at software.amazon.awssdk.utils.CompletableFutureUtils.lambda$forwardExceptionTo$0(CompletableFutureUtils.java:78)
	at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:907)
	at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:885)
	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:554)
	at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2238)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryingExecutor.maybeAttemptExecute(AsyncRetryableStage.java:135)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryingExecutor.maybeRetryExecute(AsyncRetryableStage.java:152)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryingExecutor.lambda$attemptExecute$1(AsyncRetryableStage.java:113)
	at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:907)
	at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:885)
	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:554)
	at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2238)
	at software.amazon.awssdk.utils.CompletableFutureUtils.lambda$forwardExceptionTo$0(CompletableFutureUtils.java:78)
	at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:907)
	at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:885)
	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:554)
	at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2238)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeAsyncHttpRequestStage.lambda$execute$0(MakeAsyncHttpRequestStage.java:108)
	at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:907)
	at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:885)
	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:554)
	at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2238)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeAsyncHttpRequestStage.completeResponseFuture(MakeAsyncHttpRequestStage.java:255)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeAsyncHttpRequestStage.lambda$executeHttpRequest$3(MakeAsyncHttpRequestStage.java:167)
	at java.base/java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:978)
	at java.base/java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:955)
	at java.base/java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:526)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1575)
Caused by: software.amazon.awssdk.core.exception.SdkClientException: Failed to send the request: A callback has reported failure.
	at software.amazon.awssdk.core.exception.SdkClientException$BuilderImpl.build(SdkClientException.java:111)
	at software.amazon.awssdk.core.exception.SdkClientException.create(SdkClientException.java:47)
	at software.amazon.awssdk.services.s3.internal.crt.S3CrtResponseHandlerAdapter.handleIoError(S3CrtResponseHandlerAdapter.java:192)
	at software.amazon.awssdk.services.s3.internal.crt.S3CrtResponseHandlerAdapter.handleError(S3CrtResponseHandlerAdapter.java:184)
	at software.amazon.awssdk.services.s3.internal.crt.S3CrtResponseHandlerAdapter.onFinished(S3CrtResponseHandlerAdapter.java:154)
	at software.amazon.awssdk.crt.s3.S3MetaRequestResponseHandlerNativeAdapter.onFinished(S3MetaRequestResponseHandlerNativeAdapter.java:25)

Jan 17 '25 23:01 cgoina

The current implementation that checks for an AnonympusCredentialsProvider instance is incomplete because.

I can write my own anonymous provider just like () -> MyAnonymousAWSCredentials and this will still fail
If the current AnonymousCredentialsProvider is first one in a AwsCredentialsProviderChain it still fails with the same error

Jan 18 '25 22:01 cgoina

I can't reproduce the NullPointerException.

In theory, you would not need to add the null checks in CrtCredentialsProviderAdapter because if it's using AnonymousCredentialsProvider then the code would hit return before it reaches that point.

In you code example a s3Adapter is used, does it have any custom implementation, like a custom AnonymousCredentialsProvider?

    ListObjectsV2Publisher listObjectsV2Publisher = 
        s3Adapter.getAsyncS3Client().listObjectsV2Paginator(listRequest);

Jan 27 '25 19:01 debora-ito

Another possible cause is you're using an old version of software.amazon.awssdk.crt:aws-crt, it should be 0.33.6 with SDK version 2.30.1.

Jan 27 '25 19:01 debora-ito

I am using 'software.amazon.awssdk.crt:aws-crt:0.33.9' but the example I provided will not fail because it uses AnonymousCredentials provider and the implementation already checks that. As I mentioned use a credentials provider defined as a lambda or a chain provider that has the anonymousprovider first in the chain:

            AwsCredentialsProviderChain.Builder credentialsProviderBuilder = AwsCredentialsProviderChain.builder();
            credentialsProviderBuilder
                     .addCredentialsProvider(AnonymousCredentialsProvider.create())
                    .addCredentialsProvider(ProfileCredentialsProvider.create())
                    .addCredentialsProvider(SystemPropertyCredentialsProvider.create())
                    .addCredentialsProvider(EnvironmentVariableCredentialsProvider.create())
                    .addCredentialsProvider(InstanceProfileCredentialsProvider.create());
            credentialsProvider = credentialsProviderBuilder.build();

If you put the AnonymousCredentialsProvider last in the chain it will work. If you put it first it doesn't. In some sense this is a regresssion because the async client created by S3AsyncClient.builder() works with the above chain - the one created by crtBuilder doesn't

Jan 27 '25 22:01 cgoina

Okay, I see the issue now: when the AnonymousCredentialsProvider is the first in the credential provider chain, the default s3AsyncClient will pick and use it, while the S3CrtAsyncClient will throw the error. We'll look into this.

In my local tests, putting the AnonymousCredentialsProvider as last throws the error too if none of the previous providers was picked, so I would double-check in your tests if another credential provider is being picked up before the chain reaches the AnonymousCredentialsProvider.

And as a side note, since having AnonymousCredentialsProvider as first works for s3AsyncClient, this means that the other providers in the chain will never be used. Just want to be sure this is the behavior you expected to see.

Jan 28 '25 18:01 debora-ito

I am getting the same NPE when trying to fallback to AnonymousCredentials when other credentials don't work. I have tried two options, but neither of them works with S3CrtAsyncClient.

1- I have tried to create a custom AwsCredentialProvider that wraps the DefaultCredentialsProvider and overwrites the resolveCredentials method by trying the target provider and returning AnonymousCredentials if it fails.

    @Override
    AwsCredentials resolveCredentials() {   
        try {
            return target.resolveCredentials()
        } catch (Exception e) {
            log.debug("No AWS credentials available - falling back to anonymous access")
            return AnonymousCredentialsProvider.create().resolveCredentials()
        }
    }

2- I have also tried with a chain of providers such as: AwsCredentialsProviderChain.of(DefaultCredentialsProvider.builder().build(), AnonymousCredentialsProvider.create())

The workaround I have found is to try to resolve the credentials with the default provider when configuring the client and if it fails I set the anonymous provider. However, I am not sure if it will work in all cases. Is there a plan to fix the issue to support any of the previous cases?

Jul 21 '25 19:07 jorgee

This issue is now closed. Comments on closed issues are hard for our team to see. If you need more assistance, please open a new issue that references this one.

Nov 10 '25 19:11 github-actions[bot]