flink-htm icon indicating copy to clipboard operation
flink-htm copied to clipboard

Implement event-time ordering

Open EronWright opened this issue 8 years ago • 2 comments

It is possible for events to arrive out-of-order into the HTM operator. We should reorder the events using an internal queue, using the watermark to make progress.

See the flink-cep library for an example of this: CEPPatternOperator.

EronWright avatar Apr 02 '16 23:04 EronWright

HTM. Java sensors index every element. This was preparation for parallel decoding which hasn't been turned on; but is present. Maybe you should consider going through the sensor layer because it does a lot for you?

Though I was thinking, if the data is out of order before it reaches htm.java - then Sensor indexing won't help?

cogmission avatar Apr 03 '16 00:04 cogmission

@cogmission reordering must be done at the Flink layer, using the event time information and watermarks.

EronWright avatar Apr 03 '16 19:04 EronWright