kinesis-sql
kinesis-sql copied to clipboard
Fix listShards to only specify nextToken
This fixes two bugs:
- If
listShards
returns multiple pages, then we run into an infinite loop withwhile (!nextToken.isEmpty)
, becausenextToken
will never be set back tonull
because:
if (returnedToken != null) {
nextToken = returnedToken
-
listShards
cannot allow bothStreamName
andNextToken
to be specified together.
@itsvikramagr this is tested against streams with >1k shards which forces paging on the Kinesis API
Does this read the equal number of messages too ? because I see there is a data skew.