kcl
kcl copied to clipboard
feature: start & end times for consuming
kcl can support consuming from specific timestamps.
The syntax for this should be s@<microsecond>
and e@<microsecond>
(where <microsecond>
refers to a unix microsecond timestamp), and this can be pieced into the existing -o
flag. Both or just one can be specified.
The logic for this is to basically first get the metadata for all consumed topics, and then issue a list offsets request with Timestamp being the start timestamp. While consuming, if the record timestamp for a partition surpasses the end timestamp, stop.