Popular graphana dashboards show no data for "Pods Memory usage" and "Pods CPU usage"

[surfingjoe]

Discussed in https://github.com/prometheus-operator/kube-prometheus/discussions/1694

^{Originally posted by surfingjoe March 21, 2022} Running bare-metal K8s with flannel network and MetalLB as Load Balancer. Cloned https://github.com/prometheus-operator/kube-prometheus.git just a few days ago. Created "monitoring" namespace. Performed Kubectl apply for manifests/setup, waited till all pods are up, then applied manifests/

Instead of port forwarding, I patched each service to be: "Type" load-balancer considering that I'm using MetalLB.

Everything is operational. Can successfully deploy containers with services exposed externally by the load balancer MetalLB. All is well in the world. Can open Prometheus port 9090 & Graphana port 3000 externally from a browser and everything seems operational. Graphana is seeing Prometheus data, and community dashboards are seeing some of the Prometheus data.

Import popular community dashboards like Community dashboard number 315 for example. Oops, the dashboard reports "No Data" for PODS memory and CPU usage, but Graphana can see Cluster Memory & CPU usage. The dashboard can see Network I/O pressure, but "No data" for Pods network I/O.

As I'm a newbie to Graphana, I'm hoping someone wiser than me can help :)

[MaximilianMeister]

Same issue here, it seems that some metrics referenced from the dashboard queries like node_namespace_pod_container are not available/exported by the metrics-server

[paulfantom]

There is no metric called node_namespace_pod_container. If you mean namespace_pod_container:container_cpu_usage_seconds_total:sum_rate then it was changed to namespace_pod_container:container_cpu_usage_seconds_total:sum_irate in https://github.com/kubernetes-monitoring/kubernetes-mixin. Although it might have been a mistake as per https://github.com/kubernetes-monitoring/kubernetes-mixin/issues/679. Feel free to contribute to that project as all changes from there are synced into kube-prometheus automatically.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had any activity in the last 60 days. Thank you for your contributions.

[cguertin14]

+1 on this, cpu+memory and other metrics are not showing in the provided grafana dashboards

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had any activity in the last 60 days. Thank you for your contributions.

[jehutyy]

+1 issue still there

[fguiet]

+1 issue still here as well (Kubernetes deployed with Rancher)

# Stack deployed
prometheus-stack                monitoring-ns   10              2022-12-09 12:41:56.026097892 +0100 CET deployed        kube-prometheus-stack-42.2.0            0.60.1

[greg-peters]

@fguiet, edit the panel and remove name=~"^k8s_.*", from all the panels that aren't showing data. Your current settings are filtering out the view.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had any activity in the last 60 days. Thank you for your contributions.

[weironz]

same here, kuernetes v1.25.6 installl with kubeadm , kube-prometheu-stack v0.63.0 install with helm

[DolceParadise]

Same here. Number of replicas are correctly reflected in and shown in graph but there N/A for pod cluster usage

[michaelfresco]

same here, kuernetes v1.25.6 installl with kubeadm , kube-prometheu-stack v0.63.0 install with helm

To fix this, I used the prometheus-community/kube-prometheus-stack, but with custom values.yaml;

nodeExporter:
  enabled: true
prometheus-node-exporter:
  prometheus:
    monitor:
      enabled: true

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had any activity in the last 60 days. Thank you for your contributions.

[prionkor]

Is there any fix for the issue? or any dashboard id out there that I can import. I want to see pod resource uses over time for a deployment.

[eddyzags]

+1 issue still there

[comminutus]

Same problem for me

[MahdadGhasemian]

@fguiet, edit the panel and remove name=~"^k8s_.*", from all the panels that aren't showing data. Your current settings are filtering out the view.

Thanks. It worked.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had any activity in the last 60 days. Thank you for your contributions.