[grafana] Grafana failed to load dashboard from json error=EOF

[mpavlovicbb]

Hi,

I am trying to import dasboards with helm installation but always getting following error

lvl=eror msg="failed to load dashboard from " logger=provisioning.dashboard type=file name=test file=/var/lib/grafana/dashboards/test/minio.json error=EOF

values file have following configuration:

dashboardProviders: dashboardproviders.yaml: apiVersion: 1 providers: - name: 'test' type: file disableDeletion: false editable: true allowUiUpdates: true options: path: /var/lib/grafana/dashboards/test

dashboards: test: minio: file: dashboards/minio.json

And file structure is following:

. ├── dashboards │   └── minio.json └── values.yaml

Afther initialising pod there is empty file minio.json on /var/lib/grafana/dashboards/test

Inistalation is done with command

helm upgrade --install -n grafana grafana grafana/grafana --values values.yaml

Is there any solution to this or I am missing simething?

[qdupuy]

Hello,

you have to declare your dashboard volume in the grafana container so that it can find your json files

[chukynax]

Hello,

you have to declare your dashboard volume in the grafana container so that it can find your json files

could you please clarify ?

[roshan8]

I'm running into the same issue. Any workaround?

[chukynax]

i put json next to values and set variable during setup

dashboards:
  default:
    kubernetes:
      file: dashboards/kubernetes.json

another option is to put raw json in values

dashboards:
  default:
    kubernetes:
      json:|
        {}

i wanted to use helm charts repo just to set needed values in argocd, but for these workarounds i had to fetch untar chart itself and store it in own repo. all other tricks i tried with no luck

[roshan8]

@chukynax Second option would make the file not so maintainable. Didn't get your first option though. Can you please explain what do you mean by JSON next to values?

[chukynax]

@roshan8 fetch untar grafana chart, then downloaded dashboard in json format and put it next to values.yaml in chart, in my example in directory: "dashboards"

[roshan8]

oh got it. Thanks!

[ewerton-silva00]

Hey, guys.

I have the same issue.

How are organized files:

.
├── dashboards
│   └── container-metrics.json
└── values.yaml

1 directory, 2 files

My values.yaml file:

rbac:
  create: true

adminUser: admin
adminPassword: strongpassword

persistence:
  type: pvc
  enabled: true
  storageClassName: standard
  accessModes:
    - ReadWriteOnce
  size: 10Gi

plugins:
  - grafana-clock-panel

datasources:
  datasources.yaml:
  apiVersion: 1
  datasources:
  - name: Prometheus
    type: prometheus
    url: http://prometheus.prometheus.svc.cluster.local:9090
    access: default
    isDefault: true

dashboardProviders:
  dashboardproviders.yaml:
    apiVersion: 1
    providers:
      - name: 'dojot'
        orgId: 1
        folder: 'Dojot'
        folderUid: ''
        type: file
        disableDeletion: false
        editable: false
        updateIntervalSeconds: 10
        allowUiUpdates: false
        options:
          path: /var/lib/grafana/dashboards/dojot

dashboards:
  dojot:
    container-metrics:
      file: dashboards/container-metrics.json

sidecar:
  datasources:
    enabled: true
  dashboards:
    enabled: true

I tried too without the sidecars, but I have the same result.

Command utilized to install the Helm Charts:

helm upgrade --install grafana grafana/grafana --version 6.20.3 --create-namespace --namespace grafana --values values.yaml

The result:

t=2021-12-20T12:31:28+0000 lvl=eror msg="failed to load dashboard from " logger=provisioning.dashboard type=file name=dojot file=/var/lib/grafana/dashboards/dojot/container-metrics.json error=EOF
t=2021-12-20T12:31:38+0000 lvl=eror msg="failed to load dashboard from " logger=provisioning.dashboard type=file name=dojot file=/var/lib/grafana/dashboards/dojot/container-metrics.json error=EOF

[DucretJe]

I have the same issue deploying with argocd using this repo (639fa1c) as source. My values.yaml:

        dashboardProviders:
          dashboardproviders.yaml:
            apiVersion: 1
            providers:
            - name: 'default'
              orgId: 1
              folder: ''
              type: file
              disableDeletion: false
              editable: true
              options:
                path: /var/lib/grafana/dashboards/default
        dashboards:
          default:
            kubernetes-cluster:
              gnetId: 315
              revision: 3
              datasource: Prometheus

I noticed while connecting to grafana pod and having a look in /var/lib/grafana/dashboards/default that my json file was actually empty. I looked in the logs of the download-dashboards init container and see nothing fishy

4+ set -euf
3+ mkdir -p /var/lib/grafana/dashboards/default
2+ curl -skf --connect-timeout 60 --max-time 60 -H 'Accept: application/json' -H 'Content-Type: application/json;charset=UTF-8' https://grafana.com/api/dashboards/315/revisions/3/download
1+ sed '/-- .* --/! s/"datasource":.*,/"datasource": "Prometheus",/g'

[Swaps76]

Hello,

you have to declare your dashboard volume in the grafana container so that it can find your json files

Can you please clarify - I am getting the same issue. Thanks!

I am just digging around and wondering if this is where you mean to add the volume to the value.yaml?

## Additional grafana server volume mounts
# Defines additional volume mounts.
extraVolumeMounts: []
  # - name: extra-volume-0
  #   mountPath: /mnt/volume0
  #   readOnly: true
  #   existingClaim: volume-claim
  # - name: extra-volume-1
  #   mountPath: /mnt/volume1
  #   readOnly: true
  #   hostPath: /usr/shared/

[ktomaszx]

I see many question regarding this case so I will leave this comment to clarify why this importing dashboard config from json file doesn't work. Below code dashboards: {} default: custom-dashboard: file: dashboards/custom-dashboard.json does not mean that we are importing custom-dashboard.json file from workspace we are working in. It uses custom-dashboard.json file from chart repository. https://github.com/grafana/helm-charts/blob/main/charts/grafana/dashboards/custom-dashboard.json This is why configmap data: is empty. Really bad that is not described in values.yaml comments.

[g-linville]

I think @ktomaszx is right. I ran into this same problem. I recommend using --set-file with Helm to just insert raw JSON like this:

helm install releasename grafana/grafana -f my_values.yaml --set-file dashboards.default.dashboard-name.json=my-dashboard.json

[Abuu001]

Open

For me actually I had the same problem for some weeks .

it was not easy to notice the cause but I had missed only one symbol in my dashboard.json file . After weeks of debugging I actually found out my dashboard.json file had a missing } (closing brace) at the end of file.

[trydalch]

@ktomaszx So is there any way to load a file from the workspace directory? I'm using kube-prometheus-stack and just overwriting values.yaml.

[double-yaya]

some one resolved this isson?

[jeswant]

I'm having the same issue unfortunately

grafana logger=provisioning.dashboard type=file name=qa-dashboards t=2023-03-27T12:52:16.963445325Z level=error msg="failed to load dashboard from " file=/var/lib/grafana/dashboards/qa-dashboards/general-qa-dashboard.json error=EOF

configMap (grafana-dashboards-qa-dashboards) and cat /var/lib/grafana/dashboards/qa-dashboards/general-qa-dashboard.json are empty

dashboardProviders:
 dashboardproviders.yaml:
   apiVersion: 1
   providers:
   - name: 'default'
     orgId: 1
     folder: ''
     type: file
     disableDeletion: false
     editable: true
     options:
       path: /var/lib/grafana/dashboards/default
   - name: 'qa-dashboards'
     orgId: 1
     folder: 'QA Dashboards'
     # folderUid: qa-dashboards
     type: file
     disableDeletion: false
     editable: true
     options:
       path: /var/lib/grafana/dashboards/qa-dashboards

dashboards:
  default:
    kubernetes-views-pods:
      gnetId: 15760
      datasource: Thanos
  qa-dashboards:
    general-qa-dashboard:
      # folderUid: qa-dashboards
      file: dashboards/general-qa-dashboard.json
      datasource: Thanos

folder structure looks accurate. helm upgrade --install grafana grafana/grafana -n monitoring -f grafana_values.yaml --create-namespace --version "6.52.4"

[dminca]

Have you seen this?

https://grafana.com/docs/grafana/latest/administration/provisioning/#provision-folders-structure-from-filesystem-to-grafana

Probably that's why it's not working... I see you're providing a folderUid

[jeswant]

Tested the following scenarios:

• Removed folderUid and folder. Added foldersFromFilesStructure: true in dashboardProviders/qa-dashboards. Removed folderUid from dashboards/qa-dashboards/general-qa-dashboard
• Removed foldersFromFilesStructure: true, and added folderUid and folder in dashboardProviders/qa-dashboards. Removed folderUid from dashboards/qa-dashboards/general-qa-dashboard

Continued to see:

grafana logger=provisioning.dashboard type=file name=qa-dashboards t=2023-03-29T05:25:52.513948565Z level=error msg="failed to load dashboard from " file=/var/lib/graf │ │ ana/dashboards/qa-dashboards/general-qa-dashboard.json error=EOF

Just wondering to see if this feature works atleast for someone deploying via helm charts

[jeswant]

Finally got it working with the following steps and with the values mentioned in my first comment:

helm repo add grafana https://grafana.github.io/helm-charts
helm pull grafana/grafana
helm repo remove grafana 
tar -xvzf grafana-6.52.4.tgz
cp dashboards/general-qa-dashboard.json grafana/dashboards/general-qa-dashboard.json
helm upgrade --install grafana ./grafana --version "6.52.4" -n monitoring -f grafana_values.yaml

HTH!

[PrateekKhatri]

Hi,

Anyone find solution for this, I am also trying to setup dashboards on runtime using helm. Here are my values.yaml file (file name: default-dashboards.yaml):

Folder structure:

[abiydv]

I ran into this issue trying to setup a minimal Grafana instance (with couple of custom dashboards) through ArgoCD. My workaround to deploy dashboards involves the use of kustomize.

At a high level,

• Generate configMaps using the configMapGenerator - kustomize can read the local dashboards and load the raw json as data.
• Specify this generated configMap under the dashboardsConfigMaps property in values.yaml.

I have added the example configs here if it helps anyone.

[Paul-Michaud]

You can use the same method as @abiydv using helm (Thanks for the idea)

values.yaml

grafana:
  # Omited all other variables
  dashboardProviders: 
    dashboardproviders.yaml:
      apiVersion: 1
      providers:
      - name: dashboards
        type: file
        disableDeletion: true
        editable: false
        options:
          path: /var/lib/grafana/dashboards
          foldersFromFilesStructure: true

  dashboardsConfigMaps:
    default: "mydashboards-cm"

# Create a custom variable, we follow the scheme of the dashboards variable from grafana chart
myDashboards:
  default:
    custom-dashboard:
      file: dashboards/custom-dashboard.json

Create a custom template that uses our myDashboards variable, following charts/grafana/templates/dashboards-json-configmap.yaml mydashboards-json-configmap.yaml

{{- if .Values.myDashboards }}
{{ $files := .Files }}
{{- range $provider, $dashboards := .Values.myDashboards }}
apiVersion: v1
kind: ConfigMap
metadata:
  name: mydashboards-cm
  namespace: {{ include "grafana.namespace" $ }}
  labels:
    dashboard-provider: {{ $provider }}
{{- if $dashboards }}
data:
{{- $dashboardFound := false }}
{{- range $key, $value := $dashboards }}
{{- if (or (hasKey $value "json") (hasKey $value "file")) }}
{{- $dashboardFound = true }}
  {{- print $key | nindent 2 }}.json:
    {{- if hasKey $value "json" }}
    |-
      {{- $value.json | nindent 6 }}
    {{- end }}
    {{- if hasKey $value "file" }}
    {{- toYaml ( $files.Get $value.file ) | nindent 4}}
    {{- end }}
{{- end }}
{{- end }}
{{- if not $dashboardFound }}
  {}
{{- end }}
{{- end }}
---
{{- end }}

{{- end }}

The folder looks like this

.
.
├── Chart.lock
├── charts
│   └── grafana-6.0.1.tgz
├── Chart.yaml
├── dashboards
│   └── custom-dashboard.json
├── templates
│   └── mydashboards-json-configmap.yaml
└── values.yaml

And finally, depending on your configuration you should have the file /var/lib/grafana/dashboards/default/custom-dashboard.json in the orange-grafana container of the grafana pod !

[dbaltor]

If you're deploying Grafana using kube-prometheus-stack, the easiest way to install dashboards with your deployment is just creating ConfigMaps with the grafana_dashboard label equals 1 and the data section containing the dashboard json definition as below:

apiVersion: v1
kind: ConfigMap
metadata:
  name: sample-grafana-dashboard
  labels:
     grafana_dashboard: "1"
data:
  k8s-dashboard.json: |-
  [...]

The sidecar is going to pick all your dashboards up as long as they have the correct label. Although you can chuck all dashboards under the same ConfigMap, the team's recommendation is to use one dashboard per ConfigMap. Source: https://github.com/grafana/helm-charts/tree/main/charts/grafana#sidecar-for-dashboards