Support individual file access

[moshloop]

trafficstars

There are some scenarios where being able to access a specific file in a repository is useful, for example in GrafanaDashbboard

apiVersion: integreatly.org/v1alpha1
kind: GrafanaDashboard
metadata:
  name: "dashboard-from-url"
spec:
   url: http://source-controller:9090/gitrepository/flux-system/dashboard/latest/some-dashboard.json

[hiddeco]

The GrafanaDashboard kind supports references to ConfigMap resources, which makes this not a strong argument to drastically change the way contents from Git are distributed.

[kingdonb]

https://github.com/kingdonb/flux2/blob/bcaa4c692881e0be24cd18ea44630b4d10b2b132/manifests/monitoring/monitoring-config/kustomization.yaml#L9-L11

Here is an example of how you can include a Grafana dashboard in a configmapgenerator, so that you can carry the raw grafana json files in your GitRepository and still use them directly in a Flux kustomization.

This is highly derivative of the docs example, (https://fluxcd.io/docs/guides/monitoring/) which already does this, but perhaps does not draw enough attention to it. I'm adding my own examples further down the kustomize overlay (here), where I needed dashboards to be added for ingress-nginx:

https://github.com/kingdonb/flux2/blob/bcaa4c692881e0be24cd18ea44630b4d10b2b132/manifests/monitoring/monitoring-config/kustomization.yaml#L15-L17

Please let us know if you have trouble with this technique, or if there's a more compelling argument for a change here!

(Closing)

[moshloop]

@kingdonb That method works but it covers a very narrow use case:

1. The maximum size of a config map is 1MB, Grafana dashboards can be up to 150 - 200KB, which limits this method to only a few dashboards that can be loaded, Dozens to hundreds of dashboards on a single grafana instance are not uncommon.

2. It does not support the initial use case of using the grafana operator, i.e. enabling teams to provision their own dashboards in a declarative way on a single grafana instance, without giving all teams admin access to that instance.

@scottrigby wdyt ?

[kingdonb]

While the original/canonical example uses a single configmapgenerator, the supplementary material that I provided shows how additional dashboards can each be generated and loaded in a separate configmap, so I'm not seeing how it does not meet the need that you're expressing. Would it be helpful to add this information directly to the guide?

I think there's room for a lot more documentation about how Flux and Grafana/Prometheus can all be used together. But I don't think it's likely that source-controller is going to accept this request.

As for (2), I can't say if allowing teams to load an arbitrary grafana dashboard in a configmap constitutes granting them admin access, but I can see that it's been noted:

The GrafanaDashboard kind supports references to ConfigMap resources,

It seems like if users are creating a new GrafanaDashboard for each dashboard, then creating a ConfigMap for each one is a natural fit, and showing how to do this with Flux, leveraging these features in Kustomize, would make for a great addition to the docs. What am I not understanding about this request?

[kingdonb]

I would change the issue topic to:

Support GrafanaDashboard users with documentation (about how to use with Flux)