amazon-kinesis-producer (bug) Segfault when trying to connect to Kinesis via EKS IRSA in 0.15.9

Running the producer in an EKS pod using IRSA that allows access to Kinesis.

The producer segfaults with this error:

Description: Segmentation fault
Signal: SIGSEGV Code: (1) Address: 18446744073709551608

The thought was that it was swallowing an error related to the inability to authenticate to the Kinesis endpoint. I have verified authorization is correct by using the AWS CLI on the container (which uses the EKS IRSA credential chain). I don't see any "access denied" API calls to Kinesis in Cloudtrail either. I have confirmed there is no network traffic from the container to Kinesis or Cloudwatch.

I also bumped producer logging up to TRACE, but don't see anything helpful.

Any ideas on how to further troubleshoot this?

Thanks!

Jan 29 '24 21:01 joshuabaird

After debugging and tips from this SO, it seems like there is a bug in v0.15.9 that causes the segfault. Reverting to v0.15.8 fixes the issue.

Using this to build the KPL config:

    private static WebIdentityTokenCredentialsProvider webIdentityTokenCredentialsProvider =
        WebIdentityTokenCredentialsProvider.builder()
            .roleArn(System.getenv("AWS_ROLE_ARN"))
            .roleSessionName(UUID.randomUUID().toString())
            .webIdentityTokenFile(System.getenv("AWS_WEB_IDENTITY_TOKEN_FILE"))
            .build();       

    private static final KinesisProducerConfiguration kplConfig = new KinesisProducerConfiguration()
                                                                      .setAggregationEnabled(false)
                                                                      .setRegion("us-east-1")
                                                                      .setLogLevel("trace")
                                                                      .setCredentialsProvider(webIdentityTokenCredentialsProvider);

I see that 0.15.10 is available (not sure if it fixes the issue or not) -- but it hasn't yet been released to Maven Central. It does look like the only fix in 0.15.10 was related to something that was introduced in 0.15.9, though.

I have opened AWS case #170656583201515 and provided debug/trace logs.

Jan 30 '24 20:01 joshuabaird

I'm having the same problem, thanks for the details