[FLINK-35243][cdc] Supports more schema change event types & backfill pre-schema payload for sink
This PR closes FLINK-35243:
- Supports
AlterColumnCommentEvent,AlterTableCommentEvent,DropTableEvent,RenameTableEvent, andTruncateTableEventin pipeline framework - Adds
SchemaChangeEventWithPreSchemainterface, allowing SchemaRegistry backfilling schema before change payload - Fixed MySQL pipeline source doesn't capture
ALTER TABLE ... MODIFY COLUMNDDL - Adds corresponding test cases
This requires #3339 being merged first.
@PatrickRen @ruanhang1993 PTAL
DropTableEvent TruncateTableEvent can be closed. Sometimes, tables cannot be deleted synchronously.
DropTableEvent TruncateTableEvent can be closed. Sometimes, tables cannot be deleted synchronously.
Hi @melin, thanks for your comments. It's a common use case to apply some "safe" schema change events (like AddColumnEvent) and ignore "dangerous" events (like Truncate / Drop tables). So we're designing a new feature allowing users to configure schema change behaviour for each type of schema change events in FLINK-35242. Also, we're trying to expose interfaces allowing Sink connectors to report their supported kind of schema change events.
This PR is still in early draft stage, so feel free to leave your comments if you have any other concerns.
support add/drop primary key event
Thanks for @melin's suggestion, will be working on this after #3339 got merged.
Seems it's not meaningful to rebase, since there are too many conflicts. Will open another PR to address this.