paimon icon indicating copy to clipboard operation
paimon copied to clipboard

[cdc] Support debezium avro format

Open harveyyue opened this issue 2 years ago • 10 comments

Purpose

Linked issue: close #xxx

Tests

API and Format

Documentation

harveyyue avatar Sep 25 '23 12:09 harveyyue

Can someone enable the CI test?

harveyyue avatar Sep 27 '23 09:09 harveyyue

https://github.com/harveyyue/incubator-paimon/tree/dbz-format @harveyyue This will show the result of CI

wg1026688210 avatar Sep 28 '23 09:09 wg1026688210

Hi, can you solve the conflicts?

yuzelin avatar Oct 23 '23 07:10 yuzelin

Seem this PR is not active now. I will continue to work on it.

yuzelin avatar Jan 11 '24 09:01 yuzelin

I was trying to test consuming debezium avro data using this branch. I found that it needed support for some complex types. I made changes here: https://github.com/harveyyue/incubator-paimon/pull/1/files

Not sure if they are appropriate as I had to change TypeUtils to handle complex types as well.

umeshdangat avatar Jan 13 '24 01:01 umeshdangat

@umeshdangat I'm working on merging these codes into the master branch. When it is done, you can create a PR to support your needed types.

yuzelin avatar Jan 15 '24 03:01 yuzelin

@yuzelin Sorry for late response, I'm involving in another project in past few months. Are you still working for debezium-avro format? I have drafted a pr 2783 which will be compatible with other data source or format and easy to implment the debezium-avro format.

harveyyue avatar Jan 24 '24 10:01 harveyyue

@harveyyue Glad to hear from you! I just start to this. It's good if you are willing to continue to work on this.

yuzelin avatar Jan 25 '24 07:01 yuzelin

@yuzelin @harveyyue checking in to see if this is going to be picked up soon?

umeshdangat avatar Apr 03 '24 21:04 umeshdangat

@JingsongLi @yuzelin you help merge this? it’s a blocker for @umeshdangat to start a PoC

polyzos avatar Apr 17 '24 16:04 polyzos

Thanks @harveyyue for the contribution and all for your attention!

JingsongLi avatar Aug 12 '24 00:08 JingsongLi