[promtail] allow to inject templates

[Wielewout]

In this PR the extraVolumes and initContainers can be filled in with templates in a parent chart.

For the initContainers it allows to for example inject a container that waits until kafka is ready. From the context of a parent chart, a service would be created where the name for example is {{ .Release.Name }}-kafka. This could for example be injected in kcat to check if kafka is ready.

It's also possible now to include a secret that's created in a parent chart (with the release name included in the name). This could be useful to inject a password using password_file with a file coming from a volume made with that secret.

This could for example look like the following in a values file of a parent chart. Note that {{ $.Release.Name }}-loki-gateway-auth is a secret here with usernames as keys and the corresponding password as a value within it.

promtail:
  config:
    enabled: true
    clients:
      - url: http://{{ .Release.Name }}-loki-gateway.{{ .Release.Namespace }}:80/loki/api/v1/push
        tenant_id: logs
        basic_auth:
          username: promtail
          password_file: /etc/promtail/secrets/password
  
  extraVolumeMounts:
    - name: loki-gateway-password
      mountPath: /etc/promtail/secrets

  extraVolumes:
    - name: loki-gateway-password
      secret:
        secretName: "{{ $.Release.Name }}-loki-gateway-auth"
        items:
          - key: promtail
            path: password

[CLAassistant]

All committers have signed the CLA.

[zanhsieh]

@zalegrala Just curious, who are the members of loki maintainers? This chart includes them in Chart.yaml but not win much of their attention.

[zalegrala]

I'm not sure @zanhsieh. I think folks are encouraged to use the loki chart as mentioned here: https://github.com/grafana/helm-charts/tree/main/charts/loki The promtail functionality is probably better suited to be using the alloy chart, here: https://github.com/grafana/alloy/tree/main/operations/helm/charts/alloy

I don't think there is a problem with the community maintaining extra charts here, but it would be good to get more community involvement, since don't have a test environment for much of this, so getting other community members to sanity check and approve what might break in their environment would be helpful.

[dimgogos]

Perhaps @camrossi, @Sheikh-Abubaker would be kind enough to have a look at this PR?

[Sheikh-Abubaker]

@Wielewout could you please explain the scope of this PR ?

[Wielewout]

@Wielewout could you please explain the scope of this PR ?

@Sheikh-Abubaker The main idea is to give more possibilities to integrate promtail as a dependency of a parent chart. As almost always the release name is included in resources, tpl calls are needed to get the proper value down where needed.

My specific goal is to deploy promtail in a way that it can scrape some topics on kafka and push data in loki. All of these are deployed with a single parent chart where kafka, loki and promtail are dependencies. Problem is that there were some crashes if kafka is not ready yet. Also loki is deployed with the basic auth enabled through the loki-gateway, so the additional promtail deployment needs to get credentials to be able to connect.

The tpl call for the init container allows to prevent the crashes by injecting the kafka service name (with the release name) to wait for kafka to be ready.

The tpl call for extra volumes allows to inject a secret. This secret would be created by the parent chart with a randomly generated password and is also injected in loki-gateway. This mainly prevents having an insecure default password injected directly through the values files. What this would look like in the values can be seen in the original description.

If desired I can split up the PR for each of the two tpl calls. Or I would gladly add more of them if there are other use cases that we want to cover for integration of promtail through a parent chart :smile: