Helm - allow loading all alloy files in /etc/alloy

[fculpo]

trafficstars

PR Description

This PR implements this cli feature: https://github.com/grafana/alloy/commit/0b0b4a5e2d097238f94de36c0a8722b78ab5f01b

Currently, the chart only allows to load a single alloy file as an entrypoint. This behaviour, even with modules, makes overrides or unloading components in some clusters difficult when using a common configuration.

This PR enables loading all alloy files in /etc/alloy (a la terraform). Configmap can then be overridden in some environments/clusters by not loading, adding some files, or only modifying the needed one.

Which issue(s) this PR fixes

Fixes #228

Notes to the Reviewer

PR Checklist

• [x] CHANGELOG.md updated
• [x] Documentation added

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[fculpo]

@tpaschalis I updated README with helm-docs. Sorry, forgot this.

[fculpo]

For a full explanation (discussion in Slack: https://grafana.slack.com/archives/C01050C3D8F/p1718700321152549)

This is the jsonnet code deploying Alloy (we use Tanka + Flux)

local fluxcd = import 'fluxcd.libsonnet';
local helmRelease = fluxcd.helmRelease;
local helmRepository = fluxcd.helmRepository;

local k = import 'k.libsonnet';
local configMap = k.core.v1.configMap;
local envVar = k.core.v1.envVar;

local name = 'grafana-alloy';
local namespace = 'grafana-alloy';

{
  _config:: {
    version: '<0.4',
    cluster: error 'cluster required',
    loki_url: 'https://sth/loki/api/v1/push',
    tempo_url: 'https://sth/',
    pyro_url: 'https://sth/',
    values: {
      alloy: {
        stabilityLevel: 'public-preview',
        clustering: {
          enabled: false,
        },
        configMap+: {
          create: false,
          name: $.configMap.metadata.name,
          key: 'config.river', // unneeded with PR
        },
        mounts: {
          varlog: true,
          dockercontainers: true,
          extra: [],
        },
        securityContext: {
          privileged: true,
          runAsNonRoot: false,
        },
        resources: {
          requests: {
            cpu: '10m',
            memory: '256Mi',
          },
          limits: {
            memory: '256Mi',
          },
        },
        extraEnv: [
          envVar.new('CLUSTER', $._config.cluster),
          envVar.new('LOKI_ENDPOINT_URL', $._config.loki_url),
          envVar.new('TEMPO_ENDPOINT_URL', $._config.tempo_url),
          envVar.new('PYRO_ENDPOINT_URL', $._config.pyro_url),
        ],
        envFrom: [
          {
            secretRef: {
              name: name, // ingress authentication
            },
          },
        ],
        extraPorts: [
          {
            name: 'otlp-grpc',
            port: 4317,
            targetPort: 4317,
            protocol: 'TCP',
          },
          {
            name: 'otlp-http',
            port: 4318,
            targetPort: 4318,
            protocol: 'TCP',
          },
        ],
      },
      controller: {
        hostPID: true,  // needed for java profiling
        priorityClassName: (import 'infra/main.libsonnet').priorityClasses.high.metadata.name,
      },
      configReloader: {
        resources: {
          requests: {
            cpu: '10m',
            memory: '100Mi',
          },
          limits: {
            memory: '100Mi',
          },
        },
      },
      service: {
        internalTrafficPolicy: 'Local', // route to local DS pod
      },
      serviceMonitor: {
        enabled: true,
      },
    },
  },

  // we create our configmap with multiples files
  // will be expanded as files in /etc/alloy
  configMap:
    configMap.new(name) +
    configMap.metadata.withNamespace(namespace) +
    configMap.withData({
      'config.alloy': importstr './config/config.alloy',
      'pyroscope.alloy': importstr './config/pyroscope.alloy',
      // this one will be overridden in non basic auth clusters
      'pyroscope-push.alloy': importstr './config/pyroscope-push.alloy',
    }),

  helmRepository:
    helmRepository.new(name) +
    helmRepository.metadata.withNamespace(namespace) +
    helmRepository.spec.withInterval('1h') +
    helmRepository.spec.withUrl('https://grafana.github.io/helm-charts'),

  helmRelease:
    helmRelease.new(name) +
    helmRelease.metadata.withNamespace(namespace) +
    helmRelease.spec.chart.spec.withChart('alloy') +
    helmRelease.spec.chart.spec.withVersion($._config.version) +
    helmRelease.spec.chart.spec.sourceRef.withKind('HelmRepository') +
    helmRelease.spec.chart.spec.sourceRef.withName($.helmRepository.metadata.name) +
    helmRelease.spec.chart.spec.sourceRef.withNamespace($.helmRepository.metadata.namespace) +
    helmRelease.spec.install.withCrds('CreateReplace') +
    helmRelease.spec.upgrade.withCrds('CreateReplace') +
    helmRelease.spec.withInterval('1h0m0s') +
    helmRelease.spec.withValues($._config.values) +
    // will be unneeded after PR is merged
    helmRelease.spec.withPostRenderers([
      helmRelease.spec.postRenderers.kustomize.withPatches([
        // see MR https://github.com/grafana/alloy/pull/1016
        // Seems that modules are not enough in our case
        // We load all river files (like terraform) at once, so that we can override one file easily
        {
          target: {
            version: 'v1',
            kind: 'DaemonSet',
            name: 'grafana-alloy',
          },
          patch: |||
            - op: replace
              path: /spec/template/spec/containers/0/args/1
              value: /etc/alloy  
          |||,
        },
      ]),
    ]),
}

So basically we deploy a configmap containing multiple files in /etc/alloy (standard Alloy Helm Chart usage)

Now the main issue is actually in .alloy files. For pyroscope for instance (or tempo/loki/..) we need basic auth in write components, except for some clusters. But I cannot use same write module and programmatically disable basic auth (or did not find how).

So for almost all clusters pyroscope-push.alloy is:

pyroscope.write "grafana" {
  external_labels = {
		cluster = env("CLUSTER"),
	}
  endpoint {
    url = env("PYRO_ENDPOINT_URL")
    basic_auth {
			username = "pyroscope"
			password = env("PYRO_API_KEY")
		}
  }
}

But for some others I want to push via internal service, not over ingress, hence without auth.

pyroscope.write "grafana" {
  external_labels = {
		cluster = env("CLUSTER"),
	}
  endpoint {
    url = env("PYRO_ENDPOINT_URL")
  }
}

In this particular environment, I then just have to override pyroscope-push.alloyand everything will work nicely.

Same applies to Loki/Tempo/possibly Mimir if you use Alloy for collection.

[fculpo]

Basically, this PR tries to achieve what the meta k8s monitoring helm chart does: https://github.com/grafana/k8s-monitoring-helm/blob/2e2392002d17c98c100ade51e2ecaf2fa3e70a58/charts/k8s-monitoring/templates/alloy_config/_logs_service_loki.alloy.txt#L11

If/else on non "if-able" blocks

[fculpo]

Opened proposal: https://github.com/grafana/alloy/issues/1176

[github-actions[bot]]

This PR has not had any activity in the past 30 days, so the needs-attention label has been added to it. If you do not have enough time to follow up on this PR or you think it's no longer relevant, consider closing it. The needs-attention label signals to maintainers that something has fallen through the cracks. No action is needed by you; your PR will be kept open and you do not have to respond to this comment. The label will be removed the next time this job runs if there is new activity. Thank you for your contributions!

[petewall]

I'm trying to think of the use-case around this. Specifically, I don't think it "feels" right just yet. Is there a circumstance where you'd want to load all files in a directory (/etc/alloy in this case), but also want to use the config map to create the config? Meaning, create a config using the alloy.configMap.content = "<config>, but also load additional files in the same directory. I don't even think it's actually possible to do that, since the ConfigMap is mounted at /etc/alloy.

I think a more intuitive solution might be:

alloy:
  configDir: /etc/alloy
  configMap:
    create: true
    mount: true
    key: config.alloy

configDir is always used. configMap.mount controls whether the pod and container actually try to mount the configMap, if not, you're expected to replace that mount with your own volumeMount or your own.

So a few examples:

alloy:
  configMap:
    content: "my config"

Will create the configMap, with a config.alloy, and mount it. It will run against /etc/alloy/config.alloy

alloy:
  configMap:
    create: false

Will not create the configMap, but still expects one to exist, with a config.alloy, and mount it. It will run against /etc/alloy/config.alloy

alloy:
  configMap:
    create: false
    key: null

Will not create the configMap, but still expects one to exist, and will mount it. It will run against /etc/alloy

alloy:
  configMap:
    create: false
    mount: false

Will not create the configMap or mount it. It will run against /etc/alloy. You will need to use an extra volume mount.

configMap.create=true and configMap.mount=false should be an error.

this also means that the {{- define "alloy.config-map.key" -}} needs to be updated as well.