nvidia_gpu_exporter icon indicating copy to clipboard operation
nvidia_gpu_exporter copied to clipboard

Add instance filter in Grafana dashboard

Open dennys opened this issue 1 year ago • 3 comments

Is your feature request related to a problem? Please describe. When I have several machines, it's not easy to find the correct GPU by UUID. Therefore, I try to add an instance filter in dashboard.

Describe the solution you'd like The following is my procedures, plesae consider to add it, thanks.

First, add a new Grafana dashboard variable

  1. Settings -> variables
  2. Click + New variable
  3. Assign the name (I use 'instance') and label (I use 'Instance')
  4. Change 'query types' to 'label values'
  5. Select 'instance' in 'Label' field
  6. Input '\d+.\d+.\d+.\d+:9835' in Regex (I use default port 9835)
  7. Press Apply and you have a new variable in your dashboard now.

Second, modify the GPU variable

  1. Settings -> variables
  2. Click 'gpu'
  3. Modify 'label filters', I use 'instance' = '$instance'
  4. Press Apply

And you can switch the order of the 2 variables to be more intuitive.

After saving the settings, you have 2 variables in dashboard. And when you change 'instance', the 'gpu' list should be changed.

I think the Regex is not a good idea, but if I don't add it, it shows all other instances like Node exporter, ... Please let me know if there is a better solution, thanks.

dennys avatar Jan 11 '24 08:01 dennys

Hi Dennys I've a similar requirement and I see this as a great solution to have two variables so that only when a particular host is selected, only the corresponding GPUs can be selected from the second variable.

Unfortunately, I do not see the option to modify the filter to add 'instance' = '$instance'. Please check the screenshot below and suggest. Would be really helpful.

vennelakanti2511 avatar May 07 '24 06:05 vennelakanti2511

IMG_0156

vennelakanti2511 avatar May 07 '24 06:05 vennelakanti2511

@vennelakanti2511 Sorry for late reply, this is the screen shot of Grafana 10.4. As I remember, Grafana 9.x should have the same layout. Because I don't ugprade to Grafana 11 yet, I'm not sure is it changed in Grafana 11. image

dennys avatar Jun 03 '24 23:06 dennys

Dennys, thanks for the great workaround! There yet has come another question, how do I bind the IP addresses of the instances to some certain custom hostnames? Do you have any idea?

anh193 avatar Aug 22 '24 20:08 anh193

Dennys, thanks for the great workaround! There yet has come another question, how do I bind the IP addresses of the instances to some certain custom hostnames? Do you have any idea?

I don't see hostname in the output of this exporter, maybe need to issue another feature request.

dennys avatar Aug 23 '24 04:08 dennys

Dennys, thanks for the great workaround! There yet has come another question, how do I bind the IP addresses of the instances to some certain custom hostnames? Do you have any idea?

I don't see hostname in the output of this exporter, maybe need to issue another feature request.

okay. How about binding the IPs to some manually set names as like variables.?

anh193 avatar Aug 23 '24 08:08 anh193

Dennys, thanks for the great workaround! There yet has come another question, how do I bind the IP addresses of the instances to some certain custom hostnames? Do you have any idea?

I don't see hostname in the output of this exporter, maybe need to issue another feature request.

okay. How about binding the IPs to some manually set names as like variables.?

Maybe you can try to check PromQL, I'm not sure, but maybe it has some mapping function. (I try to search "promql map values", there are some replies on stackoverflow, but need some study.)

dennys avatar Aug 23 '24 08:08 dennys

@dennys @vennelakanti2511 @anh193

I tested the change in #221 in my own Grafana, and it works like a charm.

Would you like have a try?

zhan9san avatar Aug 28 '24 06:08 zhan9san

@dennys @vennelakanti2511 @anh193

I tested the change in #221 in my own Grafana, and it works like a charm.

Would you like have a try?

It works for me, cool, thanks.

dennys avatar Aug 28 '24 07:08 dennys