seatunnel icon indicating copy to clipboard operation
seatunnel copied to clipboard

[Improve] [Engine] Move Completed Checkpoint trigger to CheckpointCoordinator

Open Hisoka-X opened this issue 2 years ago • 2 comments

Purpose of this pull request

Move Completed Checkpoint trigger to CheckpointCoordinator to avoid normal Checkpoint and completed checkpoint both trigger at same time.

Check list

  • [ ] Code changed are covered with tests, or it does not need tests for reason:
  • [ ] If any new Jar binary package adding in your PR, please add License Notice according New License Guide
  • [ ] If necessary, please update the documentation to describe the new feature. https://github.com/apache/incubator-seatunnel/tree/dev/docs
  • [ ] If you are contributing the connector code, please check that the following files are updated:
    1. Update change log that in connector document. For more details you can refer to connector-v2
    2. Update plugin-mapping.properties and add new connector information in it
    3. Update the pom file of seatunnel-dist

Hisoka-X avatar Nov 22 '22 15:11 Hisoka-X

Is there a case where we have multiple sources in the single pipeline? This causes the COMPLETED_POINT_TYPE to be triggered when a single source completes

good point, I will fix this.

Hisoka-X avatar Nov 23 '22 12:11 Hisoka-X

@ashulin Temporary solution to ensure that the task can run normally, and the fine-grained part of the task completed logic will be resolved in the future.

Hisoka-X avatar Nov 24 '22 07:11 Hisoka-X