flink-cdc icon indicating copy to clipboard operation
flink-cdc copied to clipboard

[hotfix][tests] Fix occasional pipeline E2e testcases failure

Open yuxiqian opened this issue 1 year ago • 3 comments

Currently, MySqlE2eITCase's result verification strategy is as follows: First, it keeps waiting until the last expected record received, then fetch all outputs at once and check if all expected records present.

The problem is data insertion events aren't guaranteed in sequence even with single parallelism, since MySQL INSERT statements are not strictly in order. So, any records come after the "last expected record" will not be checked, causing testcases to fail.

A failed test job: https://github.com/yuxiqian/flink-cdc/actions/runs/9015394982/job/24769924116?pr=12

This PR changes the verification logic by waiting for each expected record in sequence to fix this rarely happening CI failure.

yuxiqian avatar May 10 '24 02:05 yuxiqian

cc @lvyanquan

yuxiqian avatar May 10 '24 02:05 yuxiqian

LGTM, waiting for CI pass.

lvyanquan avatar May 10 '24 11:05 lvyanquan

Could @PatrickRen help take a look? Thanks!

yuxiqian avatar May 20 '24 02:05 yuxiqian

Fixed in #3348.

yuxiqian avatar Jun 06 '24 12:06 yuxiqian