clickhouse-operator icon indicating copy to clipboard operation
clickhouse-operator copied to clipboard

Automatic schema propagation does not propagate everything

Open macobo opened this issue 3 years ago • 1 comments

Context

This chart mentions one feature:

ClickHouse cluster scaling including automatic schema propagation

The behavior of this feature did not match my expectations.

Current behavior

From @alex-zaitsev via Slack:

Only Distributed tables and related local tables were created on a second shard [added after the fact]

This means tables which are not automatically propagated include:

  • ReplicatedMergeTree tables without Distributed tables (e.g. tables not using sharding)
  • Kafka engine tables
  • Materialized views
  • Non-replicated tables

Desired behavior

New shards should copy the full schema for everything affecting ingestion. In our usecase, this includes:

  • ReplicatedMergeTree tables without Distributed tables (e.g. tables not using sharding)
  • Kafka engine tables
  • Materialized views

Note wee'd be okay if other tables/whole schema got replicated.

As a minimum some capability to mark some tables as "to propagate" would be great.

macobo avatar Mar 07 '22 17:03 macobo

@alex-zaitsev reading the release notes of 0.19.0 I think this can be closed? Haven't tested the behavior though.

macobo avatar Aug 02 '22 06:08 macobo

It was fixed while ago

alex-zaitsev avatar Feb 29 '24 08:02 alex-zaitsev