Support for the ability to add schema files via cluster configuration

[bopohaa]

We have

apiVersion: "clickhouse.altinity.com/v1"
kind: "ClickHouseInstallation"
metadata:
  name: "simple-01"
spec:
  configuration:
    clusters:
      - name: "simple"
    settings:
      format_schema_path: config.d/
    files:
      schema.proto: |

Schema file placed in /etc/clickhouse-server/config.d/schema.proto But would like to have it in the right place - /etc/clickhouse-server/format_schemas/schema.proto To do this, you need to specify something like this in the configuration

apiVersion: "clickhouse.altinity.com/v1"
kind: "ClickHouseInstallation"
metadata:
  name: "simple-01"
spec:
  configuration:
    clusters:
      - name: "simple"
    settings:
      format_schema_path: format_schemas/
    files:
      FORMAT_SCHEMAS/schema.proto: |

[Slach]

You can use initContainer in custom podTemplate Look to https://github.com/Altinity/clickhouse-operator/blob/master/docs/chi-examples/20-protobuf-schema-example.yaml

[bopohaa]

You can use initContainer in custom podTemplate Look to https://github.com/Altinity/clickhouse-operator/blob/master/docs/chi-examples/20-protobuf-schema-example.yaml

This looks like a temporary solution. I can make my own pull request. Just not sure if that would be in line with the general policy.

[qrpike]

This example doesnt work due to the following lines: https://github.com/Altinity/clickhouse-operator/blob/master/docs/chi-examples/20-protobuf-schema-example.yaml#L46-L47

Any insight on how to fix this?

[czephyr]

Look to https://github.com/Altinity/clickhouse-operator/blob/master/docs/chi-examples/20-protobuf-schema-example.yaml

I think this has some errors. Both in practice since the volumes are not declared and the code as is won't start, both in theory since when you declare the said volumes you overwrite the conf.d folder and the InitContainer will be stuck in the while loop without the file its looking for.

I managed to write my proto schema in /var/lib/clickhouse/schema_formats/ by simply mounting a ConfigMap on the path but using the SecurityContext construct you are using in the example.

          securityContext:
            runAsUser: 101
            runAsGroup: 101
            fsGroup: 101

I'm not sure if this is a security breach since this changes the user my containers run with into the clickhouse user. Would you please clarify on the reason it's not possible to write on the /var/lib/clickhouse/ folder without the securityContext and why we should use the initContainer approach just to move a file? @Slach

[Slach]

@qrpike

This example doesnt work due to the following lines: https://github.com/Altinity/clickhouse-operator/blob/master/docs/chi-examples/20-protobuf-schema-example.yaml#L46-L47

this example works as expected

chi-protobuf-example-common-configd means configmap name which generated by clickhouse-operator so, if you change your metadata name from protobuf-example to other you need to change this name in manifest

[Slach]

@czephyr

Would you please clarify on the reason it's not possible to write on the /var/lib/clickhouse/ folder without the securityContext and why we should use the initContainer approach just to move a file?

in example I just tried to realize all inside kind: ClickHouseInstallation if you will manage external kind: ConfigMap you could mount it inside /var/lib/clickhouse/schema_formats/ directly

But, clickhouse-server and change their process UID after start to clickhouse (101) even if your container user is root So, it requires applying securityContext to allow reading .proto files inside clickhouse-server process.

[czephyr]

@Slach

chi-protobuf-example-common-configd means configmap name which generated by clickhouse-operator so, if you change your metadata name from protobuf-example to other you need to change this name in manifest

then wouldnt you need a volume definition inside template.podTemplates.spec

volumes:
    - name: chi-protobuf-example-common-configd
      configMap:
        name: name-of-configmap-generated-by-operator