Helm Chart ignores S3 server-side encryption values due to helpers.tpl

[xTrekStorex]

We use Cloudian S3 storage for our loki install via helm. To enable server-side encryption (per object) Cloudian supports the same headers as AWS S3. So I figured I could just add the sse_encrytion value to the s3 storage config and set it to true. However that value never makes it into the config map.

After frying my brain on this for several hours I finally stumbled across https://github.com/grafana/loki/blob/main/production/helm/loki/templates/_helpers.tpl#L190

For the common section in the config this template generates the storage settings based on selected values - sse_encryption is not one of them. The same applies to sse.type or sse.kms_key_id.

I found a few examples setting SSE in an AWS section for storage_config, however I don't really see how that solves the missing SSE in the common section of loki config. https://grafana.com/docs/loki/latest/configuration/examples/

Am I missing something?

Helm values:

    minio:
      enabled: false
    loki:
      storage:
        bucketNames:
          chunks: ${S3_BUCKET}
          ruler: ${S3_BUCKET}
          admin: ${S3_BUCKET}
        type: s3
        s3:
          endpoint: ${S3_ENDPOINT}
          region: ${S3_REGION}
          secretAccessKey: ${S3_SECRETKEY}
          accessKeyId: ${S3_ACCESSKEY}
          s3ForcePathStyle: false
          insecure: false
          sse_encryption: true
          sse:
            type: SSE-S3
      storage_config:
        hedging:
          at: "250ms"
          max_per_second: 20
          up_to: 3
      schemaConfig:
        configs:
          - from: "2022-01-11"
            store: boltdb-shipper
            object_store: s3
            schema: v12
            index:
              prefix: loki_index_
              period: 24h
      limits_config:
        max_streams_per_user: 100000
    read:
      extraArgs: 
        - -config.expand-env=true
      extraEnvFrom:
        - secretRef:
            name: loki-s3-credentials
    write:
      extraArgs: 
        - -config.expand-env=true
      extraEnvFrom:
        - secretRef:
            name: loki-s3-credentials
    gateway:
      enabled: true
      basicAuth:
        enabled: true
        existingSecret: loki-gateway-credentials
      ingress:
        enabled: true
        ingressClassName: nginx
        annotations:
          cert-manager.io/cluster-issuer: letsencrypt-production
        hosts:
          - host: loki.<redacted>.org
            paths:
              - path: /
                pathType: Prefix
        tls:
          - secretName: loki-gateway-tls
            hosts:
              - loki.<redacted>.org
    test:
      enabled: false
    monitoring:
      dashboards:
        enabled: false
      selfMonitoring:
        enabled: false
      lokiCanary:
        enabled: false

Resulting config map / loki config:

auth_enabled: true
common:
  compactor_address: 'loki-read'
  path_prefix: /var/loki
  replication_factor: 3
  storage:
    s3:
      access_key_id: ${S3_ACCESSKEY}
      bucketnames: ${S3_BUCKET}
      endpoint: ${S3_ENDPOINT}
      insecure: false
      region: ${S3_REGION}
      s3forcepathstyle: false
      secret_access_key: ${S3_SECRETKEY}
limits_config:
  enforce_metric_name: false
  max_cache_freshness_per_query: 10m
  max_streams_per_user: 100000
  reject_old_samples: true
  reject_old_samples_max_age: 168h
  split_queries_by_interval: 15m
memberlist:
  join_members:
  - loki-memberlist
query_range:
  align_queries_with_step: true
ruler:
  storage:
    s3:
      access_key_id: ${S3_ACCESSKEY}
      bucketnames: ${S3_BUCKET}
      endpoint: ${S3_ENDPOINT}
      insecure: false
      region: ${S3_REGION}
      s3forcepathstyle: false
      secret_access_key: ${S3_SECRETKEY}
    type: s3
runtime_config:
  file: /etc/loki/runtime-config/runtime-config.yaml
schema_config:
  configs:
  - from: "2022-01-11"
    index:
      period: 24h
      prefix: loki_index_
    object_store: s3
    schema: v12
    store: boltdb-shipper
server:
  grpc_listen_port: 9095
  http_listen_port: 3100
storage_config:
  hedging:
    at: 250ms
    max_per_second: 20
    up_to: 3
table_manager:
  retention_deletes_enabled: false
  retention_period: 0

[darrenwatt]

Ran into the same issue here. Managed to work around it using 'structuredConfig' field in the helm chart.

loki:
  structuredConfig:
    common:
      storage:
        s3:
          sse:
            type: SSE-S3
    ruler:
        storage:
          s3:
            sse:
              type: SSE-S3