flink-cdc
flink-cdc copied to clipboard
[FLINK-34883] Fix postgres uuid column as PK
A majority of Postgres databases use UUIDs as primary keys.
When we enable 'scan.incremental.snapshot.enabled = true', Flink-CDC will try to split into chunks. The splitTableIntoChunks function relies on the queryMinMax function, which fails when trying to calculate the MIN(UUID) and MAX(UUID), as that is not supported in Postgres.
This change those queries to cast the uuid to a String value.
What was done:
- added uuid type for postgres mapped as string
- for max and min queries, the uuid is casted to text: MIN(uuid::text)
- for queries with >= and < we cast the param passed as: uuid >= (?)::uuid -- this way we can still use the index and only cast one value by query
- changes on other databases are because we need to change an Splitter interface, to pass the Column instead of columnName
@GOODBOY008 can you help here to find someone that can help review this?
@leonardBang can you help here?
Thanks @eskabetxe for the contribution, I'd like to find someone who is more familiar to Pg to review this PR.
@loserwang1024 would you like to review this PR?
@GOODBOY008 any chance to get this merged?