seatunnel
seatunnel copied to clipboard
[Fix][Mongo-CDC] Fix the issue where mongo isExactlyOnce defaults to true, causing room to malfunction
Purpose of this pull request
When I was synchronizing a 500 million table of MongoDB, the snapshot phase had not yet been completed, and the cluster restarted. The task needed to be restored, which caused a memory room and CPU utilization to climb. After testing, it was confirmed that the default value of the isExactlyOnce method was true, which caused the issue.
@hailin0. thank you for your guidance!
Does this PR introduce any user-facing change?
How was this patch tested?
Check list
- [x] If any new Jar binary package adding in your PR, please add License Notice according New License Guide
- [x] If necessary, please update the documentation to describe the new feature. https://github.com/apache/seatunnel/tree/dev/docs
- [x] If you are contributing the connector code, please check that the following files are updated:
- Update plugin-mapping.properties and add new connector information in it
- Update the pom file of seatunnel-dist
- Add ci label in label-scope-conf
- Add e2e testcase in seatunnel-e2e
- Update connector plugin_config
Could you add new option named
exactly_onceto control the behavior? Just like other cdc connector.
ok! thanks!
Thanks @jw-itq for update! Please update the docs.
ok!