Docs: document how to configure storage bucket (including authentication)

[pracucci]

I've seen some questions from community about how to configure storage bucket and its authentication. We should add a documentation page to cover it, including:

• S3 authentication options (including IAM profile) and required IAM permissions (see https://github.com/grafana/mimir/discussions/2264 and https://github.com/grafana/mimir/discussions/2853#discussioncomment-3484312). Also mention how the max hops allowed by metadata service can affect it (see discussion here). Example of (very wide) S3 IAM privileges here.
• GCS authentication options and required permissions
• Azure blob storage authentication options and required permissions
  • When using the Helm chart, mention that the endpoint_suffix should be blob.core.windows.net (see comment)

References:

• How a community user configured GCS
• Issues a community user faced setting up S3 with IAM auth

[HeshamMeneisi]

@pracucci I'm also having an issue with the S3 IAM auth (IRSA). Can you share some details?

[pracucci]

We should extend the doc introduced in https://github.com/grafana/mimir/pull/2347.

[mtrin]

Hi all - I suggest clear examples on how to use structured config on helm, and how to configure Azure and\or how to config it so it uses your own StorageClass. (as well as sinppets for each implementation) I realised you use Thanos backend and had to go read their docos (which are also a bit cryptic on how it does Azure) here's how I configured using account/key

mimir:
  structuredConfig:
    multitenancy_enabled: false
    no_auth_tenant: aks_cluster_name
    common:
      storage:
        backend: azure
        azure:
          account_name: accountName
          account_key: "${STORAGE_ACCOUNT_KEY}"
          endpoint_suffix: 'blob.core.windows.net'
    blocks_storage:
      backend: azure
      azure:
        container_name: mimir-block-storage
    ruler_storage:
      backend: azure
      azure:
        container_name: mimir-ruler-storage
    alertmanager_storage:
      backend: azure
      azure:
        container_name: mimir-alertmanager-storage

I'm not to clear now on what I need to do so it doesn't need an account\key since we already have a storageclass that will provision it? or is it because it wants to create a bucket(container in Azure naming)?

[mtrin]

Ok after some digging I realised this is not supported because of the module that is used https://github.com/Azure/azure-sdk-for-go/tree/main/sdk/storage/azblob#prerequisites

Is there a way to use Azure Blob CSI Driver? As I understand it won't be possible to use this due to multi-node filesystem limitation but in that case ReadWriteMany should work?

[pracucci]

Is there a way to use Azure Blob CSI Driver? As I understand it won't be possible to use this due to multi-node filesystem limitation but in that case ReadWriteMany should work?

I'm not aware of anyone using Azure Blob CSI Driver as a replacement for the object storage in Mimir, and I'm not familiar with it.

From a quick read to the doc, it looks like it allows you to "mount" an Azure storage container as a local disk / volume. If my understanding is correct, then please continue read the following.

Generally speaking, we strongly recommend to use our client directly talking to the object storage, with no layer in between (that's what we support). That being said, if you want to give it a try, you may consider using the Mimir filesystem backend type for the object storage, and configure the filesystem path to to the path where the driver mounted the volume.

[mtrin]

@pracucci thanks a lot for the clarification