seatunnel icon indicating copy to clipboard operation
seatunnel copied to clipboard

[Improve][CDC] Improve cdc task state restore for zeta

Open hailin0 opened this issue 1 year ago • 2 comments

Purpose of this pull request

[Improve][CDC] Improve cdc task state restore for zeta

Does this PR introduce any user-facing change?

No

How was this patch tested?

Already exists

Check list

  • [ ] If any new Jar binary package adding in your PR, please add License Notice according New License Guide
  • [ ] If necessary, please update the documentation to describe the new feature. https://github.com/apache/seatunnel/tree/dev/docs
  • [ ] If you are contributing the connector code, please check that the following files are updated:
    1. Update plugin-mapping.properties and add new connector information in it
    2. Update the pom file of seatunnel-dist
    3. Add ci label in label-scope-conf
    4. Add e2e testcase in seatunnel-e2e
    5. Update connector plugin_config
  • [ ] Update the release-note.

hailin0 avatar Aug 22 '24 11:08 hailin0

It seem lack test case. Shall we add some? For example, read updated/deleted column data to unsupported transform/sink, what's will happen? And read table -> savepoint -> update column -> restore? I think the change will solve the problem of this scenario. But we need test case.

Added

https://github.com/apache/seatunnel/pull/7463/files#diff-56c06c7dde242a3733bec78c1d71f5a32baa3530be2c8f336e28de7e390312a8

hailin0 avatar Aug 27 '24 15:08 hailin0

@hailin0 hi, I think this part needs a design to help me understand the whole process

Carl-Zhou-CN avatar Sep 03 '24 06:09 Carl-Zhou-CN

@hailin0 hi, I think this part needs a design to help me understand the whole process

cc @Carl-Zhou-CN

https://github.com/apache/seatunnel/issues/7930

hailin0 avatar Oct 28 '24 07:10 hailin0

LGTM. cc @dailai for a look

Waiting to merge #7908

dailai avatar Nov 04 '24 03:11 dailai

@hailin0 hi, I think this part needs a design to help me understand the whole process

cc @Carl-Zhou-CN

#7930

He is great

Carl-Zhou-CN avatar Nov 14 '24 01:11 Carl-Zhou-CN