Support different timestamp/incrementing column name for multiple tables

[abo]

[graywill]

Also running into this issue. Legacy database uses different column names for timestamp columns. In one table it is LASTMODIFIED, in another its LAST_UPD. Not a deal breaker, can run multiple instances of Connect with different configurations. Would be nice to have the option to override the column names per table.

[sunshineo]

Is it possible to have this feature? It's been a year since the ticket open. It is a common practice on the table schema to have different incrementing column name for different table so when you do sql join query, you don't need to rename 'id' to something else.

[ekamradt]

We could really use this option. The possibility of one connector running multiple tables, with a description for each table containing timestamp and/or incrementing column name.

[vishbin]

+1 this feature could be really useful .

[marioalvial]

+1

[jpatel956]

@sunshineo , As i am new to kafka, can you explain in more detail? how to acheive syncing when single database having multiple tables having different column name.

Thanks

[markteehan]

for instance, using the TPCH schema; if connect starts with table.whitelist="" to stream all tables with mode="incrementing" then the incrementing column for each of the 8 tables could be specified as a comma delimited list of table_name.column.name, denoting the single-column incrementing column name. For instance incrementing.column.name="PART.P_PARTKEY, REGION.R_REGIONKEY, NATION.N_NATIONKEY, SUPPLIER.S_SUPPKEY , CUSTOMER.C_CUSTKEY, ORDER.O_ORDERKEY"

The code change needed is to search this list to match the table name and use the specified column name as the incrementing.column.name. This syntax does not handle composite primary keys - for instance LINEITEM which has PRIMARY KEY (L_ORDERKEY, L_LINENUMBER) - however I believe that this is not supported by the JDBC connector anyway.

[mikeakala]

Is this ever going to be supported?

[SecretServiceNinja]

"handle composite primary keys" for numeric and non-numeric columns will be available anytime soon ?