clickhouse-sink-connector
clickhouse-sink-connector copied to clipboard
Connection issue "because "conn" is null"
Connector settings:
name=xxxxxx
connector.class=com.altinity.clickhouse.sink.connector.ClickHouseSinkConnector
tasks.max=1
clickhouse.server.url=xxx
clickhouse.server.user=xxx
clickhouse.server.password=xxx
clickhouse.server.database=xxx
clickhouse.server.port=8123
topics.regex=topic_prefix\.schema\..*
store.kafka.metadata=true
topic.creation.default.partitions=1
store.raw.data=false
store.raw.data.column=raw_data
buffer.flush.time.ms=1000
thread.pool.size=1
fetch.min.bytes=10485760
enable.kafka.offset=false
replacingmergetree.delete.column=_sign
auto.create.tables=true
schema.evolution=false
table.engine=ReplacingMergeTree
deduplication.policy=off
metadata.max.age.ms=10000
key.converter=io.confluent.connect.avro.AvroConverter
value.converter=io.confluent.connect.avro.AvroConverter
key.converter.schema.registry.url=...
value.converter.schema.registry.url=...
value.converter.schemas.enable=false
key.converter.schemas.enable=false
batch.max.records=100000
sink.connector.max.queue.size=500000
Errors:
22:25:00.574 [Sink Connector thread-pool-0] ERROR com.altinity.clickhouse.sink.connector.db.DBMetadata - Error with DB connection
22:25:00.574 [Sink Connector thread-pool-0] ERROR com.altinity.clickhouse.sink.connector.db.DBMetadata - Error with DB connection
22:25:00.574 [Sink Connector thread-pool-0] ERROR com.altinity.clickhouse.sink.connector.db.DBMetadata - Error with DB connection
22:25:00.574 [Sink Connector thread-pool-0] ERROR com.altinity.clickhouse.sink.connector.executor.ClickHouseBatchRunnable - *** TABLE METADATA not retrieved for Database(careem), table(booking), retrying on next attempt
22:25:00.574 [Sink Connector thread-pool-0] ERROR com.altinity.clickhouse.sink.connector.executor.ClickHouseBatchRunnable - Error processing records for topic: storage_dbz_workbench_clickhouse_test.careem.booking
22:25:01.574 [Sink Connector thread-pool-0] ERROR com.altinity.clickhouse.sink.connector.executor.ClickHouseBatchRunnable - *** TABLE METADATA not retrieved for Database(careem), table(booking) retrying
22:25:01.574 [Sink Connector thread-pool-0] ERROR com.altinity.clickhouse.sink.connector.db.DBMetadata - Error getting alias columns
java.lang.NullPointerException: Cannot invoke "com.clickhouse.jdbc.ClickHouseConnection.createStatement()" because "conn" is null
at com.altinity.clickhouse.sink.connector.db.DBMetadata.getAliasAndMaterializedColumnsForTableAndDatabase(DBMetadata.java:379) ~[clickhouse-kafka-sink-connector-0.0.9.jar:?]
at com.altinity.clickhouse.sink.connector.db.DBMetadata.getColumnsDataTypesForTable(DBMetadata.java:310) [clickhouse-kafka-sink-connector-0.0.9.jar:?]
at com.altinity.clickhouse.sink.connector.db.DbWriter.updateColumnNameToDataTypeMap(DbWriter.java:192) [clickhouse-kafka-sink-connector-0.0.9.jar:?]
at com.altinity.clickhouse.sink.connector.executor.ClickHouseBatchRunnable.processRecordsByTopic(ClickHouseBatchRunnable.java:315) [clickhouse-kafka-sink-connector-0.0.9.jar:?]
at com.altinity.clickhouse.sink.connector.executor.ClickHouseBatchRunnable.run(ClickHouseBatchRunnable.java:189) [clickhouse-kafka-sink-connector-0.0.9.jar:?]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) [?:?]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) [?:?]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) [?:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
at java.lang.Thread.run(Thread.java:840) [?:?]`
KafkaConnect version:
{"version":"7.3.0-ccs","commit":"b8341813ae2b0444690121942f62c3a125fbf4b3","kafka_cluster_id":"xxx"}
Connect Plugins:
{
"class": "com.altinity.clickhouse.sink.connector.ClickHouseSinkConnector",
"type": "sink",
"version": "0.0.1"
},
{
"class": "com.clickhouse.kafka.connect.ClickHouseSinkConnector",
"type": "sink",
"version": "v1.2.6"
},
{
"class": "io.confluent.connect.jdbc.JdbcSinkConnector",
"type": "sink",
"version": "10.8.0"
},
{
"class": "io.debezium.connector.jdbc.JdbcSinkConnector",
"type": "sink",
"version": "2.4.2.Final"
},
{
"class": "io.confluent.connect.jdbc.JdbcSourceConnector",
"type": "source",
"version": "10.8.0"
},
{
"class": "io.debezium.connector.mongodb.MongoDbConnector",
"type": "source",
"version": "2.4.2.Final"
},
{
"class": "io.debezium.connector.mysql.MySqlConnector",
"type": "source",
"version": "2.4.2.Final"
},
{
"class": "io.debezium.connector.postgresql.PostgresConnector",
"type": "source",
"version": "2.4.2.Final"
},
{
"class": "org.apache.kafka.connect.mirror.MirrorCheckpointConnector",
"type": "source",
"version": "7.3.0-ccs"
},
{
"class": "org.apache.kafka.connect.mirror.MirrorHeartbeatConnector",
"type": "source",
"version": "7.3.0-ccs"
},
{
"class": "org.apache.kafka.connect.mirror.MirrorSourceConnector",
"type": "source",
"version": "7.3.0-ccs"
}
probably a duplicate of this one https://github.com/Altinity/clickhouse-sink-connector/issues/932