Loki cannot be added as a data source for grafana v10.2.2

[RP335]

Describe the bug Upon installing loki-stack via helm charts, and adding loki as a data source in grafana (v10.2.2), An error appears: Unable to connect to loki. However downgrading grafana to lower versions (eg. 8.3.5), it did add successfully.

To Reproduce Steps to reproduce the behavior:

1. Install loki-stack using helm charts (v2.9.11)
2. Opened grafana to check logs
3. Add loki as a data source, enter the loki-stack url

Expected behavior Data Source added successfullyl

Environment:

• Infrastructure: Kubernetes]
• Deployment tool: helm

Screenshots, Promtail config, or terminal output

[geniass]

there seems to be an open pull request for this: https://github.com/grafana/helm-charts/pull/2875

[gaddyya]

I noticed that it outputs an error that it cannot connect, but at the same time allows you to save the datasource and apply it later in the dashboard

[gawbul]

Even though I updated to the latest Helm chart version targeting Loki 2.9.3, I had to make a change in the values.yaml to set the image tag for Loki, as on inspection of the pods it was still running version 2.6.1. This fixed my issue. See here for more information.

NB: This is a temporary hack until the subchart is updated to the latest version in Loki Stack.

[romosa]

Even though I updated to the latest Helm chart version targeting Loki 2.9.3, I had to make a change in the values.yaml to set the image tag for Loki, as on inspection of the pods it was still running version 2.6.1. This fixed my issue. See here for more information.

NB: This is a temporary hack until the subchart is updated to the latest version in Loki Stack.

That was the issue for me. After upgrading loki to 2.9.3, it works fine now. helm upgrade --install loki --namespace=monitoring-system grafana/loki-stack --set loki.image.tag=2.9.3

[8ball030]

@romosa You ser are legend. Thank you.

[8ball030]

we just need to pin your suggested version right?