seatunnel icon indicating copy to clipboard operation
seatunnel copied to clipboard

[Imporve][Connector-V2] Imporve iceberg source connector

Open hailin0 opened this issue 3 years ago • 1 comments

Purpose of this pull request

  • Lock output row and update offset in pollNext method to support at-least-once
  • Add synchronized to IcebergStreamSplitEnumerator#run method to avoid update conflicts with readers access handleSplitRequest
  • Fix duplicate read splits when batch job state restore

Check list

  • [ ] Code changed are covered with tests, or it does not need tests for reason:
  • [ ] If any new Jar binary package adding in your PR, please add License Notice according New License Guide
  • [ ] If necessary, please update the documentation to describe the new feature. https://github.com/apache/incubator-seatunnel/tree/dev/docs

hailin0 avatar Sep 23 '22 12:09 hailin0

@ashulin PTAL

hailin0 avatar Sep 24 '22 08:09 hailin0