dragonfly-operator icon indicating copy to clipboard operation
dragonfly-operator copied to clipboard

Grafana Dashboard in Helm Chart doesn't work

Open maxpain opened this issue 1 year ago • 11 comments

Hello. I've deployed the dragonfly-operator Helm chart with the following values:

serviceMonitor:
  enabled: true

grafanaDashboard:
  enabled: true

But getting this error:

image

maxpain avatar Nov 03 '24 11:11 maxpain

@maxpain With Dragonfly, We don't deploy the prometheus. The issue seems to be that the datasource is misconfigured. Can you check for the same?

Thanks

Pothulapati avatar Nov 07 '24 17:11 Pothulapati

Yes, the provisioned dashboard doesn't have datasource variable

maxpain avatar Nov 07 '24 17:11 maxpain

You should be provisioning a prometheus instance right? if you already have one, add it into grafana and set it as the datasource of the dashboard.

The service monitor only configures prometheus to collect metrics 🤔

Pothulapati avatar Nov 08 '24 00:11 Pothulapati

@Pothulapati Yes, I have configured Prometheus (actually VictoriaMetrics, which is much better) for my Grafana installation. All my Grafana dashboards are declaratively provisioned via ConfigMaps. Almost all of them are from Helm charts and are working fine (CNPG, for example).

So, the Dragonfly grafana dashboard from the helm chart should be fixed.

maxpain avatar Nov 08 '24 06:11 maxpain

@maxpain Sure!

So, probably a way to pass the prometheus data source (including VictoriaMetrics) as a config through the helm chart?

Pothulapati avatar Nov 12 '24 13:11 Pothulapati

@Pothulapati I don't know how exactly, but all other helm charts with built-in grafana dashboards don't require configuring the Prometheus data source via helm values. They work out of the box.

maxpain avatar Nov 12 '24 13:11 maxpain

Here is an example: https://github.com/cloudnative-pg/grafana-dashboards/blob/main/charts/cluster/grafana-dashboard.json#L4 They use DS_PROMETHEUS variable.

maxpain avatar Nov 12 '24 13:11 maxpain

Looks like DS_Prometheus is a good answer where the user gets the option to select the datasource at the top.

Pothulapati avatar Dec 11 '24 08:12 Pothulapati

I also encountered a similar problem - no datasource selection and the entire dashboard is empty with errors. I was able to fix it locally by adding datasource selection similar to cloudnative-pg to templating.list:

- {
-   "hide": 2,
-   "name": "DS_PROMETHEUS",
-   "query": "prometheus",
-   "skipUrlSync": true,
-   "type": "constant"
- },
+ {
+   "current": {
+     "selected": false,
+     "text": "Prometheus",
+     "value": "prometheus"
+   },
+   "hide": 0,
+   "includeAll": false,
+   "label": "Datasource",
+   "multi": false,
+   "name": "DS_PROMETHEUS",
+   "options": [],
+   "query": "prometheus",
+   "queryValue": "",
+   "refresh": 1,
+   "regex": "",
+   "skipUrlSync": false,
+   "type": "datasource"
+ },

Can we try testing this solution? I think the problem is that "prometheus" query is hardcoded and all $DS_PROMETHEUS mentions are interpolated with this string. So, grafana doesn't ever try to use first available datasource with prometheus type. VictoriaMetrics users usually don't name their datasources "prometheus". For example, mine is called "VictoriaMetrics".

risenxxx avatar Dec 20 '24 10:12 risenxxx

Sure! @risen228 Can you raise a PR?

Pothulapati avatar Dec 23 '24 05:12 Pothulapati

@Pothulapati done: https://github.com/dragonflydb/dragonfly-operator/pull/279

risenxxx avatar Dec 23 '24 11:12 risenxxx