replicaServiceTemplate is selected at a shard, not the replica.

[RossRothenstine]

Using operator 0.18.4. My understanding of replicaServiceTemplate is that it will be generated per replica, not per shard. The goal is to have a service that can load balance between replicas specifically, not just the whole CHI.

I've omitted some configuration for posterity.

...
clusters:
  - name: simple-test
    templates:
      replicaServiceTemplate: replica-service-template
    layout:
      replicas:
        - name: ro
          shardsCount: 2
        - name: wo
          shardsCount: 2

serviceTemplates:
  - name: replica-service-template
    generateName: "replica-{chi}-{cluster}-{replica}"
...

This produces the following selector, for example:

$ kubectl describe service/replica-clickhouse-chart-test-simple-test-ro
...
clickhouse.altinity.com/app=chop
clickhouse.altinity.com/chi=clickhouse-chart-test
clickhouse.altinity.com/cluster=simple-test
clickhouse.altinity.com/namespace=default
clickhouse.altinity.com/replica=ro
clickhouse.altinity.com/shard=1
...

Please advise if I understand replicaServiceTemplate correctly, its goals, and whether or not it should be applying the shard to the selector.

[Slach]

thanks for reporting could you confirm you want create one service which cover all ro replicas in all shards and one service which cove all wo replicas in all shards?

You choose custom generateName and shardsCount: 2 look like operator will generate services for ro replica twice with the same name and overwrite clickhouse.altinity.com/shard

moreover look like replicaServiceNamePattern unused now in codebase

[RossRothenstine]

Correct, one service generated per replica is my desired use case.

That is my expectation of how replicaServiceTemplate should work, given the name and references in code (looking at the replica service name in namer.go). In practice it seems like the selector is selecting the last created shard.

On Tue, Apr 26, 2022, 8:38 PM Eugene Klimov @.***> wrote:

thanks for reporting could you confirm you want create one service which cover all ro replicas in all shards and one service which cove all wo replicas in all shards?

You choose custom generateName and shardsCount: 2 look like operator will generate services for ro replica twice with the same name and overwrite clickhouse.altinity.com/shard

moreover look like replicaServiceNamePattern unused now in codebase

— Reply to this email directly, view it on GitHub https://github.com/Altinity/clickhouse-operator/issues/923#issuecomment-1110499298, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAK6M7CKJXN57M7IQD3JH7LVHCZCJANCNFSM5UNQK3XQ . You are receiving this because you authored the thread.Message ID: @.***>

[sunsingerus]

Due to historical reasons replicaServiceTemplate is a template which is used to create a service which points to a particular instance of a running clickhouse (process). The origin hides in dark ages, when an instance of a clickhouse was called, guess, a replica. Right now operator does not create any service, which points to "a particular column of clickhouse instances with the same replication index" (a.k.a replica as well). Kinda confusing, yes. So, the proposition is the following:

1. Change meaning of replicaServiceTemplate to "service which points to a column of clickhouses"
2. Introduce hostServiceTemplate which would point to one particular instance of clickhouse.

Votes are welcome.

[RossRothenstine]

I see, so this the intended behavior. Thanks for the clarification.

For my use case, I will then replace the service objects after clickhouse-operator is completed with a different selector.

On Wed, Apr 27, 2022, 3:16 AM Vladislav Klimenko @.***> wrote:

Due to historical reasons replicaServiceTemplate is a template used to create service which points to a particular instance of a running clickhouse (process). The origin hides in dark ages, when one instance of a clickhouse was called, guess, a replica. Right now operator does not create any service, which points to "columns of clickhouse instances with the same replication index" (a.k.a replica as well). Kinda confusing, yes.

— Reply to this email directly, view it on GitHub https://github.com/Altinity/clickhouse-operator/issues/923#issuecomment-1110826455, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAK6M7GDS2XCHKZEE5U24BLVHEHYJANCNFSM5UNQK3XQ . You are receiving this because you authored the thread.Message ID: @.***>

[RossRothenstine]

Change meaning of replicaServiceTemplate to "service which points to a column of clickhouses"

Given that this could be a potentially backwards-compatibility breaking change, because someone is probably using replicaServiceTemplate as its original meaning, it stems to reason that hostServiceTemplate could be more appropriate here.

I have a path forward, and am not blocked by this, I just simply have to change our deployment pattern slightly at the end.

[alex-zaitsev]

Maybe we can have something like 'customServiceTemplate' that would allow to specify selectors or selector templates explicitly. But it requires some thinking