seatunnel icon indicating copy to clipboard operation
seatunnel copied to clipboard

[Fix][Mongo-CDC] Fix the issue where mongo isExactlyOnce defaults to true, causing room to malfunction

Open jw-itq opened this issue 10 months ago • 1 comments

Purpose of this pull request

image When I was synchronizing a 500 million table of MongoDB, the snapshot phase had not yet been completed, and the cluster restarted. The task needed to be restored, which caused a memory room and CPU utilization to climb. After testing, it was confirmed that the default value of the isExactlyOnce method was true, which caused the issue.

@hailin0. thank you for your guidance!

Does this PR introduce any user-facing change?

How was this patch tested?

Check list

  • [x] If any new Jar binary package adding in your PR, please add License Notice according New License Guide
  • [x] If necessary, please update the documentation to describe the new feature. https://github.com/apache/seatunnel/tree/dev/docs
  • [x] If you are contributing the connector code, please check that the following files are updated:
    1. Update plugin-mapping.properties and add new connector information in it
    2. Update the pom file of seatunnel-dist
    3. Add ci label in label-scope-conf
    4. Add e2e testcase in seatunnel-e2e
    5. Update connector plugin_config

jw-itq avatar Jun 18 '25 06:06 jw-itq

Could you add new option named exactly_once to control the behavior? Just like other cdc connector.

ok! thanks!

jw-itq avatar Jun 19 '25 02:06 jw-itq

Thanks @jw-itq for update! Please update the docs.

ok!

jw-itq avatar Jun 25 '25 00:06 jw-itq