kinesis-python
kinesis-python copied to clipboard
ShardReader retry does not back-off
This code is very helpful for understanding how to work with Kinesis Data Streams - thank you!
While sifting through it, I noticed that retries
, which controls the ShardReader
's back-off is never actually incremented as far as I can tell; this line adds 1 but never saves the incremented value:
https://github.com/NerdWalletOSS/kinesis-python/blob/2bf355173de05a38f685df0f5a6000cff9a2076a/src/kinesis/consumer.py#L54
This should work:
self.retries += 1
log.debug("Retrying get_records (#%d %ds): %s", self.retries, loop_status, exc)
While I'm here, this line: https://github.com/NerdWalletOSS/kinesis-python/blob/2bf355173de05a38f685df0f5a6000cff9a2076a/src/kinesis/consumer.py#L52
Should be (self.retries * 2) or 1
if the behavior is to (mostly) match the comment.