kube-state-metrics icon indicating copy to clipboard operation
kube-state-metrics copied to clipboard

Custom-resource-metrics non label conform annotation names

Open ulikl opened this issue 2 years ago • 5 comments

What happened:

When using the generic include of annotations in config for custom-resource-state , via

labelsFromPath:
       "**": [metadata, annotations]

with Objects having annotation key which contain characters, which are not allowed in Prometheus Metrics labels. e.g.

  metadata:
    annotations:
      field.cattle.io/creatorId: u-5cl2zk65bl
      ownerBindingsCreated: "true"

They are just exported as is. i.e.

custom_... {field.cattle.io/creatorId="u-5cl2zk65b", ...} 1

This leads to an "expected equal, got "INVALID"" error, when Prometheus tries to parse these metrics.

What you expected to happen:

I would expect the annotation names to be adapted analog to the standard kube_node_labels metrics, e.g.

custom_... {field_cattle_io_creatorId="u-5cl2zk65b", ...} 1

How to reproduce it (as minimally and precisely as possible):

Anything else we need to know?:

Environment:

  • kube-state-metrics version: 2.8.2
  • Kubernetes version (use kubectl version): 1.25.10
  • Cloud provider or hardware configuration: rke2
  • Other info:

ulikl avatar Jul 21 '23 08:07 ulikl

/assign @rexagod /triage accepted

@dgrisonnet thinks you might have a PR out to fix this?

dashpole avatar Jul 27 '23 16:07 dashpole

Checking if there was any movement on this?, we have had this issue for a while now.

kchopra456 avatar Jan 24 '24 15:01 kchopra456

I believe this should've been addressed in v2.9+. @ulikl @kchopra456 Could you specify the KSM version you were experiencing this on?

rexagod avatar Feb 25 '24 20:02 rexagod