Can't watch custom resources with the same kind but different API groups

[branden]

This might be another duplicate of #1848, but I'm not sure if this behavior is covered by that issue.

What happened:

I ran kube-state-metrics with the following custom resource state config, which defines two metrics for two different resource kinds Certificate that belong to different API groups:

spec:
  resources:
    - metricNamePrefix: kube_managedresource
      groupVersionKind:
        group: acm.aws.upbound.io
        version: v1beta1
        kind: Certificate
      commonLabels: {}
      labelsFromPath: {}
      metrics:
        - name: uid
          help: The UID of a acm.aws.upbound.io/v1beta1 Certificate.
          each:
            type: Info
            gauge: null
            stateSet: null
            info:
                labelsFromPath:
                    uid:
                        - metadata
                        - uid
                path: []
                labelFromKey: ""
          commonLabels: {}
          labelsFromPath: {}
          errorLogV: 0
      errorLogV: 0
      resourcePlural: certificates
    - metricNamePrefix: kube_managedresource
      groupVersionKind:
        group: acmpca.aws.upbound.io
        version: v1beta1
        kind: Certificate
      commonLabels: {}
      labelsFromPath: {}
      metrics:
        - name: uid
          help: The UID of a acmpca.aws.upbound.io/v1beta1 Certificate.
          each:
            type: Info
            gauge: null
            stateSet: null
            info:
                labelsFromPath:
                    uid:
                        - metadata
                        - uid
                path: []
                labelFromKey: ""
          commonLabels: {}
          labelsFromPath: {}
          errorLogV: 0
      errorLogV: 0
      resourcePlural: certificates

kube-state-metrics failed to start and logged the following error:

"Parsing from Custom Resource State Metrics file failed" err="found multiple custom resource configurations for the same resource certificates"

What you expected to happen:

I expected kube-state-metrics to successfully parse the file and watch the different types.

How to reproduce it (as minimally and precisely as possible):

Run kube-state-metrics with the above custom resource state config.

Anything else we need to know?:

Environment:

• kube-state-metrics version: v2.7.0
• Kubernetes version (use kubectl version): 1.23
• Cloud provider or hardware configuration:
• Other info:

[branden]

I have a branch here that fixes this bug: https://github.com/branden/kube-state-metrics/tree/watch-types-with-same-kind However it introduces a breaking change to config or CLI arguments that reference CRDs. Config that refers to a CRD with kind Thing and API group example.com as things would need to be updated to refer to it as things.example.com.

Some work would be needed to make the change backwards compatible. But if there's interest, I could open a PR and start collecting feedback and iterating.

[dgrisonnet]

Closing as a duplicated of https://github.com/kubernetes/kube-state-metrics/issues/1848.

/close

[k8s-ci-robot]

@dgrisonnet: Closing this issue.

In response to this:

Closing as a duplicated of https://github.com/kubernetes/kube-state-metrics/issues/1848.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[dgrisonnet]

Actually this is not similar since #1848 is an RFE when this is an actual bug.

@rexagod could you perhaps have a look?

/triage accepted /reopen

[k8s-ci-robot]

@dgrisonnet: Reopened this issue.

In response to this:

Actually this is not similar since #1848 is an RFE when this is an actual bug.

@rexagod could you perhaps have a look?

/triage accepted /reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[rexagod]

ACK! /assign @rexagod

[clementblaise]

I ran into the same issue to instrument ProviderConfig for multiple providers; definitely needed to better monitor Crossplane.

[SleepyBrett]

I have a branch here that fixes this bug: https://github.com/branden/kube-state-metrics/tree/watch-types-with-same-kind However it introduces a breaking change to config or CLI arguments that reference CRDs. Config that refers to a CRD with kind Thing and API group example.com as things would need to be updated to refer to it as things.example.com.

Some work would be needed to make the change backwards compatible. But if there's interest, I could open a PR and start collecting feedback and iterating.

I like this change maybe having some way to deal with both 'things' and 'things.example.com' seamlessly?

[Serializator]

I was able to reproduce the issue on the v2.8.2 tag but not the main branch. I pinpointed the commit which "solved" or at least masked the issue to https://github.com/kubernetes/kube-state-metrics/commit/25a1d8da.

@rexagod, as the committer, do you maybe know more about the status of this issue?