Support automatic table schema evolution

[mlivirov]

trafficstars

Description

In my organization we use clickhouse as analitical database that gets it's data from multiple sources via Kafka. Domain models change from time to time as business changes and we found a need to reflect those changes in the clickhouse manually a little inconvenient.

Would be awesome if this connector could propagate changes in schemas to the clickhouse tables automatically.

Additional context Compatibility modes in kafka:
https://docs.confluent.io/cloud/current/sr/fundamentals/schema-evolution.html#compatibility-types

JDBC connector that has this functionality: https://github.com/confluentinc/kafka-connect-jdbc

[Paultagoras]

Hi @mlivirov ! So this seems like it would have a race condition, if there are multiple tasks running - is that not the case with the jdbc connector mentioned?

[mlivirov]

Not sure how it's done in jdbc connector. Wouldn't it be possible to use separate KeeperMap table to synchronize tasks?