compliantkubernetes-apps icon indicating copy to clipboard operation
compliantkubernetes-apps copied to clipboard

Drop metrics with lot's of series that are unused

Open OlleLarsson opened this issue 1 year ago • 0 comments

Is your feature request related to a problem? Please describe. We have this epic for defining what metrics we should keep in ck8s. In the mean time while it is being worked on we want to remove some low hanging metrics that are generating a lot of series already now and are unused, if there are any such metrics that is.

Kubelet/cAdvisor and ingress-nginx are two components that comes to mind that are have a lot of series.

Describe the solution you'd like Browse Prometheus in wc and sc, and check what jobs generate a lot of series, then digest that down what metrics those are for. Once you have the metrics, check if they are unused. If unused, update the metricRelabelings to drop said metrics.

Describe alternatives you've considered NA

Additional context Clearly, this is only relevant for charts that have the option to configure metricRelabelings.

Also timebox this, if there aren't any obvious heavy series generating metrics that can be dropped, that is OK.

Definition of done: Some low hanging, heavy series generating metrics have been removed.

OlleLarsson avatar Oct 19 '23 07:10 OlleLarsson