amazonica icon indicating copy to clipboard operation
amazonica copied to clipboard

[kinesis] Allow fully manual checkpointing

Open tomconnors opened this issue 8 years ago • 5 comments

For my use case, neither checkpointing every n seconds nor checkpointing by returning true from the processing function is fine-grained enough. I suggest the changes in this PR. Note that I haven't tested this code - wanted to see whether you're open to this change before I spend any actual time on it.

tomconnors avatar Oct 12 '17 22:10 tomconnors

Cleaned up the things you had questions about. Now *checkpointer* is always bound when processor is called, and each checkpoint strategy is explicitly handled, except for :manual, which is handled by doing nothing.

tomconnors avatar Oct 13 '17 14:10 tomconnors

So is this ready to be merged?

mcohen01 avatar Oct 18 '17 00:10 mcohen01

With this latest commit it's working for me and I'd say it's ready to be merged. The version of the aws sdk that amazonica uses doesn't define a UserRecord.getEncryptionType method, so I just removed that line. I didn't see anywhere else in the codebase that :encryption-type was referenced.

tomconnors avatar Oct 19 '17 19:10 tomconnors

Anything I can change here to get this merged?

tomconnors avatar Mar 28 '18 15:03 tomconnors

@mcohen01 ?

dijonkitchen avatar May 16 '19 17:05 dijonkitchen