kinsumer icon indicating copy to clipboard operation
kinsumer copied to clipboard

Allow users to use ListShards to check Kinesis stream readiness

Open StevenMcVicker opened this issue 5 years ago • 3 comments

This addresses #42. Adds a boolean configuration field for toggling DescribeStream and ListShards in the kenesisStreamReady method that is called in Run

This might not be an ideal solution, but just putting it out there for consideration.

StevenMcVicker avatar Sep 16 '19 21:09 StevenMcVicker

Why make this a toggle and not replace DescribeStream directly with ListShards?

garethlewin avatar Sep 16 '19 23:09 garethlewin

Hmm, after reading this why do you propose using ListShards to tell if the stream is ready? Is the assumption that ListShards will return ResourceInUseException if kinesis is not ready?

garethlewin avatar Sep 16 '19 23:09 garethlewin

@GarethLewin, I worked with @StevenMcVicker on this. We chose to do a toggle just to try to keep everyone happy. Happy to remove it.

We chose ListShards mainly because it seems to have the highest allowed TPS (100 tps) of any of the APIs so it would hopefully mean running into the LimitExceededException less often.

And yes, the idea is that ResourceInUseException means the stream is not in an active state. This isn't one for one with describe stream, as kinesisStreamReady allows for an UPDATING state as well. From my investigation it looks like getting the StreamStatus is only possible through DescribeStream.

connormckelvey avatar Sep 17 '19 15:09 connormckelvey

travis is broken, I reviewed and tested this PR in my environment

slydon avatar Mar 15 '24 19:03 slydon