aws-sdk-java-v2 icon indicating copy to clipboard operation
aws-sdk-java-v2 copied to clipboard

DynamoDB CompletableFutures may hang forever if there is an exception in retry policy checks (DynamoDbAsyncClient + RetryCondition)

Open jocull opened this issue 9 months ago • 0 comments

Describe the bug

When a CompletableFuture<?> is requested out of the DynamoDbAsyncClient, (such as a request to software.amazon.awssdk.enhanced.dynamodb.DynamoDbAsyncTable#getItem) the future will never return if a RetryCondition check throws an exception (such as NullPointerException).

References:

  • software.amazon.awssdk.services.dynamodb.DynamoDbAsyncClient
  • software.amazon.awssdk.core.retry.conditions.RetryCondition
  • software.amazon.awssdk.enhanced.dynamodb.DynamoDbAsyncTable#getItem

Regression Issue

  • [ ] Select this option if this issue appears to be a regression.

Expected Behavior

  • Complete the original future with the exception thrown by the retry policy, or...
  • At the very minimum log the exception! Right now it is lost forever with no trace 😭

Current Behavior

  • The future returned from the DynamoDbAsyncTable#getItem or similar request never completes. It will hang forever.
  • If you don't have manual timeout wrappers around this then it may hang processing threads forever.

Reproduction Steps

  1. Configure a DynamoDbAsyncClient
  2. Supply the client with a RetryCondition
    • This condition needs to throw an exception when it is called
  3. Perform a request that fails for some reason and will check the RetryCondition

Possible Solution

No response

Additional Information/Context

We discovered this because there was a legitimate error in our configuration code. The retry check was throwing a NullPointerException, but there was no trace of it. Nothing to stdout, nothing to any logs that we could find. It took over a week to locate.

AWS Java SDK version used

2.28.1, 2.31.15

JDK version used

openjdk version "17.0.14" 2025-01-21

Operating System and version

macOS (Sequoia 15.3.2), Amazon Linux 2023 (2023.7.20250331)

jocull avatar Apr 04 '25 20:04 jocull