lxd
lxd copied to clipboard
Update metrics.md Changed /common/tls to /current/tls for metrics.crt, metrics.key & server.crt location
Changed: /var/snap/prometheus/common/tls/ to /var/snap/prometheus/current/tls/ as the location for the metrics.crt, metrics.key & server.crt files in Prometheus.
I was getting no data in Grafana after following this page.
Looked into my syslog and found the following message:
2024-05-11T03:08:48.449690+00:00 lxd-metrics prometheus.prometheus[2973]: ts=2024-05-11T03:08:48.448Z caller=manager.go:212 level=error component="scrape manager" msg="error creating new scrape pool" err="error creating HTTP client: unable to load specified CA cert /var/snap/prometheus/86/tls/server.crt: open /var/snap/prometheus/86/tls/server.crt: no such file or directory" scrape_pool=lxd
So I made a copy of server.crt in the /86/tls/ folder where the message states it was missing from and that error went away.
Then started seeing this next error regarding metrics.crt & metrics.key:
2024-05-11T03:10:31.336446+00:00 lxd-metrics prometheus.prometheus[3051]: ts=2024-05-11T03:10:31.335Z caller=manager.go:212 level=error component="scrape manager" msg="error creating new scrape pool" err="error creating HTTP client: unable to use specified client cert (/var/snap/prometheus/86/tls/metrics.crt) & key (/var/snap/prometheus/86/tls/metrics.key): open /var/snap/prometheus/86/tls/metrics.crt: no such file or directory" scrape_pool=lxd
Also moved metrics.crt & metrics.key into /86/tls/ and that second error went away.
As a test, I removed metrics.key, metrics.crt & server.crt from the original /common/tls/ location and that did not cause a problem.
It appears that Prometheus needs those .crt & .key files in the /var/snap/prometheus/86/tls folder rather than /var/snap/prometheus/common/tls , otherwise I get those errors about them being missing.
/current/ is a symlink to /86/ , so I figured there must be a reason for that and used /current/ in my edit instead of /86/
And, finally, in case this makes any difference, here is my setup:
Using Ubuntu 22.04 as my host LXD 5.21 server, I followed this page and setup Prometheus, Grafana, Loki & Promtail inside an Ubuntu 24.04 container. Proxied web access to Grafana through HAProxy.