kube-state-metrics icon indicating copy to clipboard operation
kube-state-metrics copied to clipboard

If a container doesn't have cpu limits, kube_pod_resource_limit reports the init-container limit

Open andresm53 opened this issue 1 year ago • 3 comments

What happened: Given the following pod:

apiVersion: v1
kind: Pod
metadata:
  name: example
  labels:
    app: nginx
spec:
  containers:
    - name: nginx
      image: nginx:latest
      ports:
        - containerPort: 8080
  initContainers:
  - name: init-myservice
    image: busybox:1.28
    resources:
      limits:
        cpu: 500m
        memory: 64Mi
      requests:
        cpu: 50m
        memory: 64Mi

kube_pod_resource_limit reports 500m as the pod limit.

What you expected to happen: As per Init Containers documentation:

The Pod's effective request/limit for a resource is the higher of:

  • the sum of all app containers request/limit for a resource
  • the effective init request/limit for a resource

Since the app container doesn't have cpu limits, which means "no limit", I would have expected that kube_pod_resource_limit reports none.

How to reproduce it (as minimally and precisely as possible):

  1. Create a pod using the example pasted above.
  2. Query kube_pod_resource_limit cpu resource: sum(kube_pod_resource_limit{resource='cpu',pod='example',namespace='test'})

Anything else we need to know?:

Environment: Openshift 4.12.

  • kube-state-metrics version:
sh-4.4$ ./kube-state-metrics --version
kube-state-metrics, version v2.6.0 (branch: rhaos-4.12-rhel-8, revision: 18c2653)
  build date:       2023-08-23T20:55:33Z
  go version:       go1.19.10 X:strictfipsruntime
  platform:         linux/amd64
  • Kubernetes version (use kubectl version): I0102 12:41:29.597851 1 server.go:254] "Run with Kubernetes cluster version" major="1" minor="25"
  • Cloud provider or hardware configuration: AWS

andresm53 avatar Jan 02 '24 19:01 andresm53

/assign @rexagod /triage accepted

dashpole avatar Jan 11 '24 17:01 dashpole

FYI While this seems like a bug, it is recommended to use kube-scheduler's exposed metrics for kube_pod_resource_{limit/request}s.

rexagod avatar Jan 17 '24 13:01 rexagod

Thanks @rexagod . The problem with that (kube-scheduler's exposed metrics) is: in my particular case, I am using Openshift (4.12) and by default it uses kube_pod_resource_limit to display the cpu metrics chart. This is how it looks like, for the example pod I provided before. As you can see the chart is confusing, because the pod doesn't effectively has any cpu limits, but the chart implies that it has.

293755369-a544a531-6460-4bfe-81c7-29edefd2d926

andresm53 avatar Jan 22 '24 19:01 andresm53

@andresm53 Oh wow! Thank you for bringing this up, I'll ping the console folks internally to take a look. That being said, I believe https://github.com/openshift/console would be a better place to raise this.

Closing, feel free to reopen in openshift/console.

rexagod avatar Feb 25 '24 19:02 rexagod