[Bug] Issue with setting up Temporal helm package with certificates

[valerian-martin-tbc]

Hi,

Im having an issue setting up the Temporal Helm chart with SSL certificates

Currently I have this setup for ssl(certificates have been generated by the tls-simple option from https://github.com/temporalio/samples-server/tree/main/tls):

    tls:
      enabled: true
      internode:
        server:
          certFile: /cert-dir/client.pem
          keyFile: /cert-dir/client.key
          requireClientAuth: false
          clientCaFiles:
            - /cert-dir/ca.cert
        client:
          serverName: temporal-internode.example.com
          rootCaFiles:
            - /cert-dir/ca.cert
      frontend:
        server:
          enabled: true
          certFile: /cert-dir/cluster.pem
          keyFile: /cert-dir/cluster.key
          requireClientAuth: false
          clientCaFiles:
            - /cert-dir/ca.cert
        client:
          serverName: temporal.example.com
          rootCaFiles:
            - /cert-dir/ca.cert

Everything is getting up except for the create-temporal-namespace part from temporal-schema Job(and I suspect its because it tries to connect to the temporal frontend port 7233 and since there are no certificates, it fails). Same is for the web part - getting a 500 issue due to it not connecting to the frontend port 7233.

Also if I run: tctl --address temporal.example.com:7233 --tls_cert_path client.pem --tls_key_path client.key --tls_ca_path ca.cert namespace list Im getting: transport: authentication handshake failed: tls: first record does not look like a TLS handshake

Note: Certificates are in place for the worker,frontend and history part(cannot find options for the web part where to set them up) from a mounted volume. Pretty sure this either has to be documented better and the part with the create-temporal-namespace part from temporal-schema Job needs to also somehow to be able to use these certificates - from what I see from the chart it doesn't make any difference between a setup using TLS and one that doesn't(by the way - without TLS it works fine).

[lsu-tc]

Hi, I was running into these issues previously as well when trying to set it up through the helm chart so commenting in case it could help you out. I was configuring mTLS between the components so it might be a little different.

To set up certificates for the web component, you need to specify additionalEnvs under web in the values. In my case, I specified TEMPORAL_TLS_ENABLED, TEMPORAL_TLS_CA, TEMPORAL_TLS_KEY, TEMPORAL_TLS_CERT, and TEMPORAL_TLS_SERVER_NAME but you can find the full list of envs here.

For the createNamespace Job, it also needs the same envs for the CLI to be configured with the certificate arguments. Currently theres no way to specify these in the values but I have a PR up to hopefully make that change. But if you are able to modify the schema yourself, you can put these envs in under that job and it should work.

[luismacosta]

@lsu-tc @valerian-martin-tbc

I've created a PR to generate TLS certs via cert-manager, using my own tls.crt + tls.key https://github.com/temporalio/helm-charts/pull/663

[spanktar]

For anybody else running into the same issue, this is the config that eventually worked for me to get the namespace create job to run with TLS:

  values:
    admintools:
      additionalEnv:
        - name: TEMPORAL_TLS_ENABLED
          value: "true"
        - name: TEMPORAL_TLS_CA
          value: /etc/temporal/certs/ca/ca.crt
        - name: TEMPORAL_TLS_CERT
          value: /etc/temporal/certs/client/tls.crt
        - name: TEMPORAL_TLS_KEY
          value: /etc/temporal/certs/client/tls.key
        - name: TEMPORAL_TLS_SERVER_NAME
          value: temporal-frontend.main.svc
        - name: TEMPORAL_TLS_ENABLE_HOST_VERIFICATION
          value: "true"
      additionalVolumes:
        - name: temporal-ca-cert
          secret:
            secretName: temporal-ca-cert
        - name: temporal-client-cert
          secret:
            secretName: temporal-client-cert
      additionalVolumeMounts:
        - name: temporal-ca-cert
          mountPath: /etc/temporal/certs/ca
          readOnly: true
        - name: temporal-client-cert
          mountPath: /etc/temporal/certs/client
          readOnly: true

...

    server:
      config:
        namespaces:
          # Enable this to create namespaces
          create: true
          namespace:
            - name: default
              retention: 30d

This assumes you've set up the other server TLS stuff already, but that's well documented. What was missing for me was the proper ENV var names.