kinesis-python icon indicating copy to clipboard operation
kinesis-python copied to clipboard

ShardReader retry does not back-off

Open inventhouse opened this issue 2 years ago • 1 comments

This code is very helpful for understanding how to work with Kinesis Data Streams - thank you!

While sifting through it, I noticed that retries, which controls the ShardReader's back-off is never actually incremented as far as I can tell; this line adds 1 but never saves the incremented value:

https://github.com/NerdWalletOSS/kinesis-python/blob/2bf355173de05a38f685df0f5a6000cff9a2076a/src/kinesis/consumer.py#L54

This should work:

    self.retries += 1
    log.debug("Retrying get_records (#%d %ds): %s", self.retries, loop_status, exc)

inventhouse avatar Sep 08 '22 21:09 inventhouse

While I'm here, this line: https://github.com/NerdWalletOSS/kinesis-python/blob/2bf355173de05a38f685df0f5a6000cff9a2076a/src/kinesis/consumer.py#L52

Should be (self.retries * 2) or 1 if the behavior is to (mostly) match the comment.

inventhouse avatar Sep 09 '22 10:09 inventhouse