flink-connectors icon indicating copy to clipboard operation
flink-connectors copied to clipboard

Watermark support with the new FLIP-167 API

Open crazyzhou opened this issue 3 years ago • 0 comments

Problem description In FLIP-167, a new watermark API is introduced in both the new and old Sink API We can

  • Change our current watermark implementation in SinkFunction implementation. We can get rid of the periodic checking per event and only propagate the watermark to Pravega in the writeWatermark call.
  • For new Sink API, add the watermark propagation logic in the SinkWriter and let user use an enableWatermarkPropagation(true) call to trigger in the builder.

Problem location the new and old Sink API

Suggestions for an improvement

crazyzhou avatar Dec 14 '21 02:12 crazyzhou