kcl icon indicating copy to clipboard operation
kcl copied to clipboard

feature: start & end times for consuming

Open twmb opened this issue 4 years ago • 0 comments

kcl can support consuming from specific timestamps.

The syntax for this should be s@<microsecond> and e@<microsecond> (where <microsecond> refers to a unix microsecond timestamp), and this can be pieced into the existing -o flag. Both or just one can be specified.

The logic for this is to basically first get the metadata for all consumed topics, and then issue a list offsets request with Timestamp being the start timestamp. While consuming, if the record timestamp for a partition surpasses the end timestamp, stop.

twmb avatar Oct 02 '20 06:10 twmb