Loki-simple-scalable object storage values ambiguous

[samox73]

When deploying loki-simple-scalable, the S3 configuration does not provide enough error handling and the doc is not clear about what values to set. Using

common:
  ....
  storage:
    s3:
      bucketnames: mybucketname
      s3: s3://ACCESS_KEY:URI_ENCODED_SECRET_KEY@eu-central-1

results in loki-write pods which report:

level=info ts=2022-03-25T12:50:39.699229502Z caller=table_manager.go:171 msg="uploading tables"
level=info ts=2022-03-25T12:51:39.699329652Z caller=table_manager.go:171 msg="uploading tables"
level=info ts=2022-03-25T12:51:39.703260241Z caller=checkpoint.go:617 msg="starting checkpoint"
level=info ts=2022-03-25T12:51:39.703491468Z caller=checkpoint.go:342 msg="attempting checkpoint for" dir=/var/loki/wal/checkpoint.000001
level=info ts=2022-03-25T12:51:39.706892492Z caller=checkpoint.go:504 msg="atomic checkpoint finished" old=/var/loki/wal/checkpoint.000001.tmp new=/var/loki/wal/checkpoint.000001
level=info ts=2022-03-25T12:52:39.698819226Z caller=table_manager.go:171 msg="uploading tables"

when in fact no objects are stored in the corresponding bucket.

[samox73]

using

common:
  ....
  storage:
    s3:
      access_key_id: ACCESS_KEY
      bucketnames: mybucketname
      endpoint: s3.eu-central-1.amazonaws.com
      region: eu-central-1
      s3forcepathstyle: true
      secret_access_key: URI_ENCODED_SECRET_KEY

does not seem to work either

EDIT: just to be clear, I am using the configuration from https://github.com/grafana/helm-charts/tree/main/charts/loki-simple-scalable#configuration but just changed the necessary bits to point to the S3 bucket. Querying with grafana works, since the values are cached locally in the loki-write pods, but the blobs are never actually arriving at the bucket.

EDIT2: for completeness, here's the whole config file that gets mapped to /etc/loki/config/config.yaml:

auth_enabled: false
common:
  path_prefix: /var/loki
  replication_factor: 1
  ring:
    kvstore:
      store: memberlist
  storage:
    s3:
      access_key_id: ACCESSKEY
      bucketnames: BUCKETNAME
      endpoint: s3.eu-central-1.amazonaws.com
      insecure: true
      region: eu-central-1
      secret_access_key: SECRET
limits_config:
  enforce_metric_name: false
  max_cache_freshness_per_query: 10m
  reject_old_samples: true
  reject_old_samples_max_age: 168h
memberlist:
  join_members:
  - 'loki-loki-simple-scalable-memberlist'
schema_config:
  configs:
  - from: "2021-01-01"
    index:
      period: 24h
      prefix: loki_index_
    object_store: s3
    schema: v11
    store: boltdb-shipper
server:
  http_listen_port: 3100

[pbn4]

@samox73 Did you manage to get it to work?

[samox73]

@pbn4 yes, here's the configuration I used. It's taken from a pulumi project, that's why its typescript, but should transfer to YAML:

values: {
    loki: {
        config: {
            common: {
                storage: {
                    filesystem: null,
                    s3: {
                        endpoint: "s3.eu-central-1.amazonaws.com",
                        insecure: true,
                        bucketnames: bucketname,
                        region: "eu-central-1",
                        access_key_id: accessKeyId,
                        secret_access_key: secretAccessKey,
                    },
                }
            },
            schema_config: {
                configs: [
                    {
                        from: "2021-01-01",
                        store: "boltdb-shipper",
                        object_store: "s3",
                        schema: "v11",
                        index: {
                            period: "24h",
                            prefix: "loki_index_"
                        }
                    },
                ]
            },
        },
    },
},

[pbn4]

Thank you. This saved me a lot of time. :)