Issue in shardIteratorType

[katyayanaanand]

I saw events lying in my kinesis consumer which were never getting consumed since multiple days and they would be lost after retention time so I decided to change consumer property for shardIteratorType from default LATEST to TRIM_HORIZON. But my idle events present in queue are still not consumed, but new events are getting consumed properly.

kinesis.bindings.consumeMessage-in-0.consumer.checkpointMode=manual, kinesis.bindings.consumeMessage-in-0.consumer.listenerMode=batch, kinesis.bindings.consumeMessage-in-0.consumer.shardIteratorType=TRIM_HORIZON, kinesis.binder.kplKclEnabled=true

I got following in the logs :

The [ShardConsumer{shardOffset=KinesisShardOffset{iteratorType=TRIM_HORIZON, sequenceNumber='null', timestamp=null, stream='message_queue', shard='shardId-000000000000', reset=false}, state=NEW}] has been started.

Got an exception java.lang.NullPointerException during [ShardConsumer{shardOffset=KinesisShardOffset{iteratorType=TRIM_HORIZON, sequenceNumber='null', timestamp=null, stream='message_queue', shard='shardId-000000000000', reset=false}, state=CONSUME}] task invocation. Process will be retried on the next iteration.

Here are some more details : Java 8, normal kinesis consumer and not fan out.

dependencies : 'org.springframework.boot:spring-boot-starter-web' 'org.springframework.boot:spring-boot-starter-actuator' 'io.micrometer:micrometer-registry-influx' 'org.springframework.cloud:spring-cloud-stream-binder-kinesis:2.2.0' 'com.amazonaws:aws-java-sdk-kinesis:1.12.376' 'com.amazonaws:amazon-kinesis-client:1.14.9'

[artembilan]

Would be great if you can share more info about that NullPointerException. I mean what is exactly stack trace.

Plus, keep in mind that 2.2.x version of this project is out of support.

On the other hand I wonder if this is related to old fixed issue: https://github.com/spring-cloud/spring-cloud-stream-binder-aws-kinesis/issues/187. Therefore would you mind trying this in your project: https://github.com/spring-cloud/spring-cloud-stream-binder-aws-kinesis/issues/187#issuecomment-1422983453 ?