scylla-operator
scylla-operator copied to clipboard
Prometheus metrics relabel is not up to date
What happened?
Scylla Enterprise versions 2023.1.x and higher do not show shard-level metrics in the dashboards.
What did you expect to happen?
NA
How can we reproduce it (as minimally and precisely as possible)?
NA
Scylla Operator version
NA
Kubernetes platform name and version
Kubernetes platform info:
Please attach the must-gather archive.
NA
Anything else we need to know?
Look at the metrics relabel config between this line: https://github.com/scylladb/scylla-monitoring/blob/1ab337a505e4f57ec15001480d0898c6d18c8a76/prometheus/prometheus.yml.template#L75 And this line: https://github.com/scylladb/scylla-monitoring/blob/1ab337a505e4f57ec15001480d0898c6d18c8a76/prometheus/prometheus.yml.template#L126
@amnonh - do we need to update the monitoring in the operator? 4.6 is out, it's a good time to do it?
I'm removing myself as assigned, assign me if you need more information or if you want to me to do anything
@amnonh @mykaul's directions were that monitoring team should be updating the dashboard definitions here, that's the reason this was assigned to you.
I've opened the issue around the Prometheus relabel config, which is broken for 2023.1 and higher.