grafana dashboards K8s Node doesn't display ? metrics

[shiyiwenv]

Everything else is normal. What's the reason?

[tangjiaxing669]

me too.

[sosoriov]

Same here! I checked the metrics and panel is link to node_cpu metric which is not available in my Prometheus,

[wguto]

ye it is an old metric, check my PR, it is fixed there. The list I couldnt figure out why it is not showing =[

[sensay-nelson]

@wguto oddly, using your PR I have the list working, but none of the other metrics :( . I can swear that this data was working the other day though, and then stopped working as I fixed the Container metrics.

 node-exporter:
    Image:         quay.io/prometheus/node-exporter:v0.16.0

Kubernetes:
Server Version: version.Info{Major:"1", Minor:"8", GitVersion:"v1.8.7", GitCommit:"b30876a5539f09684ff9fde266fda10b37738c9c", GitTreeState:"clean", BuildDate:"2018-01-16T21:52:38Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}

Kubernetes plugin version 1.0.1

edit: I found the source of my issue was that somewhere along the line I had deleted and re-imported the dashboards, but they picked up the old specs rather than the new ones. I'm chalking it up to Grafana's odd way of handling dashboard versions.

[sosoriov]

I would say it depends on your Prometheus configuration. Definitely the metrics had change and now you must use the new names, however, depends on your configuration you will get different labels in your metrics, for instance. I saw in @wguto PR that you filter based on instance label but in my case I have kubernetes_node label. So I would suggest @sensay-nelson that double check the values that comes from Prometheus and compare them with your Grafana template.

Some examples of the new metrics names here

https://prometheus.io/docs/guides/node-exporter/

Before it was node_loadxxx now the metric is node_loadxxx_total check the Prometheus guidelines

[sensay-nelson]

it's all a bit mysterious, but I agree that the metric name and the node label appear to shift a bit depending on setups. Using node-exporter v0.16.0 the metric names from Wguto's request were correct (node_cpu_seconds_total), but the '$instance' labels are '$node' labels in my setup.

[shiyiwenv]

it's all a bit mysterious, but I agree that the metric name and the node label appear to shift a bit depending on setups. Using node-exporter v0.16.0 the metric names from Wguto's request were correct (node_cpu_seconds_total), but the '$instance' labels are '$node' labels in my setup.

How to setup '$instance' labels are '$node' labels ？ my image update to node-exporter v0.16.0 ，That's it (node_cpu_seconds_total) 。

[wguto]

@sensay-nelson @sosoriov Another thing that I notice when working with other dashboards for more in depth network stuff, like the dashboard 1860 on the grafana repo, it is better to use the node-exporter V0.15.2. For some reason it has LOTS of more metrics for heavy network stuff like tcp stuff. Now I switched back to v0.15.2 and this dashboard is still working, of course I have to change the metrics all again to the old format.