kafka-connect-jdbc

kafka-connect-jdbc copied to clipboard

Limit Database Query to Configurable Number of Rows

13

**Idea** Add a configurable limit to a number of rows to the database query. **Context** We use the JDBC source connector on a large database table with a million new...

philipschm1tt

[2022-02-03 17:14:45,496] INFO JdbcDbWriter Connected (io.confluent.connect.jdbc.sink.JdbcDbWriter) connect | [2022-02-03 17:14:45,822] INFO Checking MySql dialect for existence of TABLE "ORCLCDB"."C__MYUSER"."EMP" (io.confluent.connect.jdbc.dialect.GenericDatabaseDialect) connect | [2022-02-03 17:14:45,862] INFO Using MySql dialect TABLE "ORCLCDB"."C__MYUSER"."EMP"...

kriskrishna

got wrong , while creating connector

1

{"error_code":500,"message":"Failed to find any class that implements Connector and which name matches io.confluent.connect.jdbc.JdbcSourceConnector, available connectors are: PluginDesc{klass=class org.apache.kafka.connect.file.FileStreamSinkConnector, name='org.apache.kafka.connect.file.FileStreamSinkConnector', version='6.0.0-ccs', encodedVersion=6.0.0-ccs, type=sink, typeName='sink', location='file:/usr/share/java/kafka/'}, PluginDesc{klass=class org.apache.kafka.connect.file.FileStreamSourceConnector, name='org.apache.kafka.connect.file.FileStreamSourceConnector', version='6.0.0-ccs', encodedVersion=6.0.0-ccs, type=source,...

Karthik7287

Sink Connector with Snowflake - Tables not Found

2

Hi, I've setup Kafka locally and I'm trying to use the JDBC Sink Connector to write data into Snowflake but I keep getting the error below suggesting that the table...

brunocostalopes

New version of Postgresql driver can not support partitioned tables

5

I upgraded our Confluent Community Package helm cluster from 5.4.1 to 6.0.1, and installed jdbc plugin: ``` confluent-hub install --no-prompt --component-dir /usr/share/confluent-hub-components/ confluentinc/kafka-connect-jdbc:10.2.0 ``` However, the new cluster can not...

echang0929

Add jar file to artifacts

5

Having a zip/tgz of the source code files is nice, but it would be really nice if you provided a `.jar` file in the artifacts for direct download so that...

QuinnBast

JDBC to hive

1

Hello, I want to use hive to connect to kafka-connect, but I have only found a way to connect to hdfs namenode. I don’t know if JDBC can be used...

GreenplumDB

Kafka Connect - GET connector api exposes password in cleartext

10

Kafka-Connect Rest api exposes OracleDB password in cleartext. I am using the latest confluent images (5.1.0) and externalized OracleDB passwords for Connect configurations in vault. I am able to successfully...

samsaida

JDBC Source using query is not fetching any records for Mariadb

8

>Here is connector configuration. ``` name=tx_user_nw_source connector.class=io.confluent.connect.jdbc.JdbcSourceConnector connection.url=jdbc:mysql://localhost:3306/?user=user&password=password&autoReconnect=true&useSSL=false&useUnicode=yes&characterEncoding=UTF-8&useLegacyDatetimeCode=false&serverTimezone=EST5EDT mode=timestamp query=SELECT A.USER_ID, C.TX_ID, C.MAST_SUB_ID, C.MAST_ID, C.STATUS_CODE, EXTRACTVALUE(P.USER_XML, '/CAT/SUB_CAT[@name=\"source\"]') AS CS_SOURCE, EXTRACTVALUE(P.USER_XML, '/CAT/SUB_CAT[@name=\"DIV\"]') AS DIV, EXTRACTVALUE(P.USER_XML, '/CAT/SUB_CAT[@name=\"SUB_DIV\"]') AS SUB_DIV C.COMMENT, C.CREATE_DATE, C.CREATE_DATETIME,...

sandycomp

Serialize across multiple (master-detail) tables

4

Any chance this Connector could serialize a message containing [ ... ] across a hierarchy of tables? Clearly UPSERT semantics would have to be replaced with DELETE (and Delete Cascade...

robinroos