kafka-connect-jdbc icon indicating copy to clipboard operation
kafka-connect-jdbc copied to clipboard

Timestamp mode not working and table.whitelist does not allow to poll data from multiple tables for jdbc Kafka connector.

Open nik09541 opened this issue 6 years ago • 2 comments

Hi All I am still facing issue with the mode timestamp in jdbc kafka source connector. All of the above recommendations have been implemented by me. I have converted the timestamp column to UTC format even using

alter table TEMP add UPDATED_DATE TIMESTAMP(6) default SYS_EXTRACT_UTC(CURRENT_TIMESTAMP) not null;

Following is the connector config

name=TRANSACTION connector.class=io.confluent.connect.jdbc.JdbcSourceConnector tasks.max=10 connection.url=jdbc:oracle:thin:@service:1701:CONNECTOR connection.user=TRANSACTION connection.password=txn#789 mode=timestamp timestamp.column.name=UPDATED_DATE validate.non.null=false #incrementing.column.name=ID #query=select * from temp inner join tempdata on temp.id=tempdata.id topic.prefix=test batch.max.rows=3 timestamp.delay.interval.ms=3000 table.whitelist=TEMP table.poll.interval.ms=3000

Issues being faced

  1. Kafka connector not polling data and not pushing to topic when I use timestamp mode or timestamp+incrementing mode , however only incrementing mode works fine.
  2. I am not able to use multiple tables to poll out data from them by mentioning them in table.whitelist separated by comma like this table.whitelist=TEMP,TEMPDATA connector always picks up second table and creates a topic with prefix 'test' and fullname testTEMPDATA. In case i mention auto.create.topics.enable=true , then nothing happens in logs and no error.

Please help over these issues as I might be missing on something. Urgent help is required .

Thanks in Advance!

nik09541 avatar Nov 01 '18 08:11 nik09541

db My db:

nik09541 avatar Nov 01 '18 08:11 nik09541

Hi, I have the same problem in RedShift.Have this issue been solved? Can you please share the solution? Thanks!!!!

dc-lilu avatar Nov 24 '21 05:11 dc-lilu