kube-state-metrics icon indicating copy to clipboard operation
kube-state-metrics copied to clipboard

feat: Add actual pod resources metrics

Open wardady opened this issue 5 months ago • 15 comments

What this PR does / why we need it: Starting from Kubernetes 1.33, kube_pod_container_resource_requests and kube_pod_container_actual_resource_limits reflect the desired state of the container resource requests/limits. This PR adds 2 new metric families: kube_pod_container_actual_resource_requests and kube_pod_container_actual_resource_limits that expose values from status.containerStatuses[i].resources and reflect the current actual requests and limits of each container.

How does this change affect the cardinality of KSM: The PR adds 2 new metric families with the same cardinality as very similar already existing metric families.

Which issue(s) this PR fixes Fixes #2665

wardady avatar Jul 08 '25 18:07 wardady

CLA Signed

The committers listed above are authorized under a signed CLA.

  • :white_check_mark: login: wardady / name: Hermann Yavorskyi (0c297ac0b3d7a6c402b67521502ff047d9e2e423, 422ba2ec77e473c8dc1b198e219f5622f251fd64, 609342e17baa5668ff6b97eac9663d536a34b490)

Welcome @wardady!

It looks like this is your first PR to kubernetes/kube-state-metrics 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes/kube-state-metrics has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. :smiley:

k8s-ci-robot avatar Jul 08 '25 18:07 k8s-ci-robot

/triage accepted /assign @CatherineF-dev

dgrisonnet avatar Aug 07 '25 16:08 dgrisonnet

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: wardady Once this PR has been reviewed and has the lgtm label, please ask for approval from catherinef-dev. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot avatar Aug 07 '25 21:08 k8s-ci-robot

@dgrisonnet @logicalhan I adjusted the failing test and now it passes locally. Could you please approve the CI run?

wardady avatar Aug 07 '25 21:08 wardady

@dgrisonnet @logicalhan would appreciate any review or feedback on this PR.

wardady avatar Aug 11 '25 10:08 wardady

When will kube_pod_container_resource_requests and kube_pod_container_actual_resource_requests have different values?

CatherineF-dev avatar Aug 13 '25 17:08 CatherineF-dev

When will kube_pod_container_resource_requests and kube_pod_container_actual_resource_requests have different values?

I assume this might be the case with https://kubernetes.io/blog/2025/05/16/kubernetes-v1-33-in-place-pod-resize-beta/ ?

mrueg avatar Aug 13 '25 19:08 mrueg

@mrueg, right, that is the case with in-place pod resize. @CatherineF-dev For example, when you try to apply an in-place pod resize patch, which changed requests of a container to the value above node capacity or available resources:

kubectl patch pod  pod-name --subresource resize --patch '{"spec":{"containers":[{"name":"container-name", "resources":{"requests":{"memory":<above-node-capacity>}}}]}}'

In this case, the value of kube_pod_container_resource_requests will be different from kube_pod_container_actual_resource_requests (kube_pod_container_resource_requests reports an updated value in this case, even though the container resources were not patched)

wardady avatar Aug 13 '25 21:08 wardady

Hey, just checking up if anything else is missing to get the review for this PR?

wardady avatar Aug 19 '25 13:08 wardady

As I've opened https://github.com/kubernetes/kube-state-metrics/pull/2773 here my 2 cents:

  • kube_pod_container_status_resource_* would be more consistent with the other metric names for container status metrics (see kube_pod_container_status_last_terminated_exitcode , etc.)
  • Extract the ResourceList handling into a function to avoid duplicating this code for the 6th time - unless you have differences (which could be valid given resizing is only supported for CPU & Memory, but likely doesn't hurt)
  • Add support for init containers as sidecars can be resized too

mrIncompetent avatar Sep 16 '25 17:09 mrIncompetent

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

k8s-ci-robot avatar Sep 17 '25 06:09 k8s-ci-robot

@CatherineF-dev @mrueg Do you happen to have some cycles to provide some feedback?

mrIncompetent avatar Sep 18 '25 13:09 mrIncompetent

Ping @CatherineF-dev @mrueg

mrIncompetent avatar Oct 08 '25 16:10 mrIncompetent

Ping @CatherineF-dev @mrueg

mrIncompetent avatar Nov 03 '25 09:11 mrIncompetent