opentelemetry-collector-contrib icon indicating copy to clipboard operation
opentelemetry-collector-contrib copied to clipboard

Exporter/clickhouse support for distributed table

Open nengc opened this issue 1 year ago • 3 comments

Component(s)

exporter/clickhouse

Is your feature request related to a problem? Please describe.

I cannot find anything about the distributed table in the codes. So cannot support sharding at this moment? https://clickhouse.com/docs/en/engines/table-engines/special/distributed

Describe the solution you'd like

Support distributed table like below: CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] ( name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], ... ) ENGINE = Distributed(cluster, database, table[, sharding_key[, policy_name]]) [SETTINGS name=value, ...]

Describe alternatives you've considered

No response

Additional context

No response

nengc avatar Apr 29 '24 09:04 nengc

Pinging code owners:

  • exporter/clickhouse: @hanjm @dmitryax @Frapschen @SpencerTorres

See Adding Labels via Comments if you do not have permissions to add labels yourself.

github-actions[bot] avatar Apr 29 '24 09:04 github-actions[bot]

@SpencerTorres closed by https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/31920?

mshustov avatar Apr 30 '24 07:04 mshustov

Hi @mshustov ,i mean the clickhouse sharding using the “Distributed Table”,not the replica

nengc avatar May 08 '24 03:05 nengc

@nengc I think cluster_name and table_engine in https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/clickhouseexporter#configuration-options wil achive your goal

Frapschen avatar May 30 '24 07:05 Frapschen

Please see this response.

The other comment is also a good suggestion if you want to use the existing config options, but for now this can be handled by manually creating the table.

By manually creating the table you will have full control over these settings. We can close this issue for now. Thanks!

SpencerTorres avatar Jun 18 '24 05:06 SpencerTorres

This can be closed @crobert-1. Users now have full control over table creation with example DDL provided

SpencerTorres avatar Aug 20 '24 14:08 SpencerTorres

Thanks @SpencerTorres!

crobert-1 avatar Aug 20 '24 15:08 crobert-1