flink-cdc icon indicating copy to clipboard operation
flink-cdc copied to clipboard
verified publisher icon apache

[FLINK-35072][cdc][doris] Support applying compatible `AlterColumnTypeEvent` to Doris sink

Open yuxiqian opened this issue 1 year ago • 2 comments

According to Doris documentation, altering column types dynamically is supported (via ALTER TABLE ... MODIFY COLUMN statement) when lossless conversion is available. However, now Doris pipeline connector has no support for any AlterColumnTypeEvents, and raises RuntimeException all the time.

This PR adds alterColumn method which is not provided by Doris' SchemaChangeManager to apply AlterColumnTypeEvent to Doris sink. Only lossless conversions that were accepted by Doris will be allowed, and a RuntimeException will be raised for any incompatible narrowing casts.

yuxiqian avatar Apr 10 '24 08:04 yuxiqian

I'm not familiar with Doris, could @lvyanquan @JNSimba please take a look?

yuxiqian avatar Apr 12 '24 02:04 yuxiqian

Since @qg-lin is implementing this change in upstream doris-flink-connector, I'll mark this as draft for now.

yuxiqian avatar Jun 19 '24 06:06 yuxiqian

Since it has been implemented in https://github.com/apache/doris-flink-connector/pull/408, I'll close it for now and bump dependency version to address this after Doris Flink connector 1.6.2 released.

yuxiqian avatar Jul 11 '24 01:07 yuxiqian