twitchscience
kenesisStreamReady produces LimitExceededException when calling DescribeStream
We are experiencing very frequent LimitExceededExceptions when starting our kinsumer applications. This is a result of the hard 10 TPS rate limit on DescribeStream calls in AWS.
Right now we are avoiding this by creating a kinsumer with the NewWithInterfaces constructor and passing in a custom kinesisiface.KinesisAPI with a workaround implementation of DescribeStream that calls ListShards instead.
It would be great if there was a config toggle to use an alternate method to check kenesis stream readiness so that we can avoid this issue.
Hi, @StevenMcVicker I'll take a look soon, but are you using the latest? I thought that https://github.com/twitchscience/kinsumer/pull/21 addressed this (if it doesn't I'm just going by the description of the issue and it might not be related)
