flink-cdc icon indicating copy to clipboard operation
flink-cdc copied to clipboard

[fix-issue-2676] repair a snapshot-split bug:

Open AidenPerce opened this issue 1 year ago • 6 comments

when use splitOneUnevenlySizedChunk, sometimes(for example, a database or a table or a column use 'utf8mb4_general_ci') we will get a big chunk. For example when the value of primaryKey like ['0000','1111','2222','3333','4444','aaaa','bbbb','cccc','ZZZZ',...] almost all of the values which starts with a [a-zA-z] will be split into a big chunk;

AidenPerce avatar Jan 05 '24 06:01 AidenPerce

Hi @AidenPerce, could you please rebase this PR with latest master branch before it could be merged? Renaming like com.ververica.cdc to org.apache.flink.cdc might be necessary.

yuxiqian avatar Apr 26 '24 07:04 yuxiqian

Hi @AidenPerce, could you please rebase this PR with latest master branch before it could be merged? Renaming like com.ververica.cdc to org.apache.flink.cdc might be necessary.

I have rebase the branch of this pr, but there are some problems in test-case of db2-connector, that caused i can't update this pr. I don't know why the case couldn't pass, this is the check log: https://github.com/AidenPerce/flink-cdc-connectors/actions/runs/8963238039/job/24613239338

AidenPerce avatar May 06 '24 03:05 AidenPerce

Hi @AidenPerce, sorry about the inconvenience. I think it's related to a glitch in Db2 incremental connector and should be fixed by #3283. Will try to get it merged asap.

yuxiqian avatar May 06 '24 05:05 yuxiqian

Hi @AidenPerce, the Db2 CI fix has been merged into master branch, could you please rebase this PR and see if this problem persists? Thank you!

yuxiqian avatar May 06 '24 05:05 yuxiqian

Hi @AidenPerce, the Db2 CI fix has been merged into master branch, could you please rebase this PR and see if this problem persists? Thank you!

I get a new problem, there is a tcase of mongodb with a failure: NewlyAddedTableITCase.testRemoveAndAddCollectionsOneByOne:330->testRemoveAndAddCollectionsOneByOne:501 expected:<15> but was:<16> like https://github.com/AidenPerce/flink-cdc-connectors/actions/runs/8965612344/job/24619438529

But i re-run the test-case successfully. Are there any issues with this test case?

AidenPerce avatar May 06 '24 08:05 AidenPerce

@yuxiqian may help to merge it please ?

AidenPerce avatar May 08 '24 03:05 AidenPerce

Hi @AidenPerce, is there any updates on this PR? Feel free to comment here if you need any help.

yuxiqian avatar Jul 12 '24 06:07 yuxiqian

This pull request has been automatically marked as stale because it has not had recent activity for 60 days. It will be closed in 30 days if no further activity occurs.

github-actions[bot] avatar Sep 11 '24 00:09 github-actions[bot]