Altinity embed 1.0

[ctaity-treinta]

Hi, i configured the emebed connector to replicate a database from mysql to clickhouse, but i am getting the following error with the _version field:

90486 2023-09-19 14:04:00.170 [pool-95-thread-6] INFO com.altinity.clickhouse.sink.connector.db.DbWriter  - *** QUERY***insert into highlighted_products(id,type,weight,product_id,location_id,created_at,updated_at,deleted_at,_version,is_deleted) select id,type,weight,product_id,location_id,created_at,updated_at,deleted_at,_version,is_deleted from input('id String,type String,weight Nullable(Int32),product_id String,location_id Nullable(Int32),created_at DateTime64(3),updated_at DateTime64(3),deleted_at Nullable(DateTime64(3)),_version UInt64,is_deleted UInt8')
90488 2023-09-19 14:04:00.172 [pool-95-thread-6] ERROR com.altinity.clickhouse.sink.connector.db.DbWriter  - ******* ERROR inserting Batch *****************
java.sql.SQLException: Cannot set null to non-nullable column #9 [_version UInt64]
	at com.clickhouse.jdbc.SqlExceptionUtils.clientError(SqlExceptionUtils.java:73)
	at com.clickhouse.jdbc.internal.InputBasedPreparedStatement.addBatch(InputBasedPreparedStatement.java:340)
	at com.altinity.clickhouse.sink.connector.db.DbWriter.addToPreparedStatementBatch(DbWriter.java:476)
	at com.altinity.clickhouse.sink.connector.executor.ClickHouseBatchRunnable.flushRecordsToClickHouse(ClickHouseBatchRunnable.java:198)
	at com.altinity.clickhouse.sink.connector.executor.ClickHouseBatchRunnable.processRecordsByTopic(ClickHouseBatchRunnable.java:169)
	at com.altinity.clickhouse.sink.connector.executor.ClickHouseBatchRunnable.run(ClickHouseBatchRunnable.java:101)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)

What can i do ?

Thanks

[subkanthi]

Hi @ctaity-treinta , Was this table created manually, is there anyway you can share the schema of the MySQL table and the docker tag you were testing.

[ctaity-treinta]

No, the table was created by the connector, the mysql schema is:

CREATE TABLE `highlighted_products` (
  `id` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
  `type` enum('HIGHLIGHT','DISCOUNT') CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT 'DISCOUNT',
  `weight` int DEFAULT NULL,
  `product_id` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
  `location_id` int DEFAULT NULL,
  `created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
  `updated_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
  `deleted_at` timestamp NULL DEFAULT NULL,
  PRIMARY KEY (`id`),
  KEY `FK_b9a6b5d78530c86ceab8782312a` (`product_id`),
  KEY `FK_ed2c693a3f73f1312c1339180c4` (`location_id`),
  CONSTRAINT `FK_b9a6b5d78530c86ceab8782312a` FOREIGN KEY (`product_id`) REFERENCES `products` (`id`),
  CONSTRAINT `FK_ed2c693a3f73f1312c1339180c4` FOREIGN KEY (`location_id`) REFERENCES `locations` (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci

[aadant]

@ctaity-treinta did you manage to workaround the issue ? which version of MySQL is it ? I wonder what could cause the _version field to be null ? Any idea @subkanthi ?

[ctaity-treinta]

no i didnt , we are using kafka connector with red panda. We tried with two versions of mysql: 5.7 and 8