I don't see label filters on k8s.node.cpu.usage metrics when using kubeletstats receiver

[lazyboson]

Component(s)

receiver/kubeletstats

Describe the issue you're reporting

I am using opentelemetry collector which is running as deamonset on eks cluster. I am using kubeletstats receiver to send node and pod's cpu stats to vendor backend. The data is availble on the grafana. But I am facing this issue that when i am trying to use - k8s_node_cpu_usage metrics, i can't filter it based on a label(which should be node id), in order to visualise each node cpu usages. Further, All the stats i receive for pods doesn't have any label associated with it, how will we know what node or what pod this metrics is? its super confusing.

here is my collector configuration -

  receivers:
    otlp:
      protocols:
        http:
    loki:
      protocols:
        http:
      use_incoming_timestamp: true
    kubeletstats:
      collection_interval: 20s
      auth_type: "serviceAccount"
      endpoint: "${env:K8S_NODE_IP}:10250"
      insecure_skip_verify: true
      metrics:
        k8s.node.cpu.usage:
          enabled: true
        k8s.node.memory.available:
          enabled: true
        k8s.node.memory.usage:
          enabled: true
        k8s.pod.cpu.usage:
          enabled: true
        k8s.pod.memory.usage:
          enabled: true

thanks

[github-actions[bot]]

Pinging code owners:

• receiver/kubeletstats: @dmitryax @TylerHelmuth @ChrsMark

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[ChrsMark]

At the moment, the receiver does not add Node's labels and annotations at first place: https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/27909

Did you try to add the k8sattributesprocessor in your pipeline? That should be enough to add labels/annotations in Pod's metrics (not sure if covers the Node's metrics though 🤔 ).

[lazyboson]

@ChrsMark Thanks for you reply. So atleast node metrics are of limited use. Off topic, is there any receivers in collector which does that?

[ChrsMark]

@ChrsMark Thanks for you reply. So atleast node metrics are of limited use. Off topic, is there any receivers in collector which does that?

I haven't tried that but maybe enabling the kubeletstats receiver through a receiver_creator could give you the Node's labels.

[lazyboson]

Both of the suggetion didn't work. anyways they are in beta stability, so can't complain. Thanks.