kube-state-metrics icon indicating copy to clipboard operation
kube-state-metrics copied to clipboard

feat: Use PartialObjectMetadata for Configmaps

Open mrueg opened this issue 1 year ago • 9 comments
trafficstars

What this PR does / why we need it: This PR introduces a metadataonly client, which can be used to fetch sparse data for large objects like configmaps.

How does this change affect the cardinality of KSM: (increases, decreases or does not change cardinality) None

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged): Fixes https://github.com/kubernetes/kube-state-metrics/issues/2463

mrueg avatar Aug 06 '24 15:08 mrueg

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mrueg

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot avatar Aug 06 '24 15:08 k8s-ci-robot

We can't use partialobjectmetadata for secrets unfortunately, since we have metrics for the secrets type that we expose.

mrueg avatar Aug 06 '24 20:08 mrueg

We can't use partialobjectmetadata for secrets unfortunately, since we have metrics for the secrets type that we expose.

Secret supports type as a field selector.

I wonder if we could make a sort of informer that does a metadata-only watch per known Secret type, plus one that watches for unknown types and doesn't use partial object metadata. Should be quite a bit more efficient overall, at the cost of turning one watch into n.

sftim avatar Aug 07 '24 16:08 sftim

/assign /assign @richabanker /triage accepted

dgrisonnet avatar Aug 08 '24 16:08 dgrisonnet

I still lack seeing the hoped memory improvements with this PR. My test looks like the following (creates 100x 1MB sized configmaps):

for i in $(seq 1 100); do
        base64 /dev/urandom | head -c 1000000 > file.txt
        kubectl create configmap morecm$i --from-file="file.txt"
done

mrueg avatar Aug 08 '24 16:08 mrueg

Few comments, but I am a fan of this change! :)

Great to hear! I'm still looking for better ideas to verify that it saves network traffic and reduces memory consumption on ksm.

mrueg avatar Aug 16 '24 11:08 mrueg

The Kubernetes project currently lacks enough contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

  • Mark this PR as fresh with /remove-lifecycle stale
  • Close this PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot avatar Feb 04 '25 21:02 k8s-triage-robot

The Kubernetes project currently lacks enough active contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

  • Mark this PR as fresh with /remove-lifecycle rotten
  • Close this PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

k8s-triage-robot avatar Mar 06 '25 21:03 k8s-triage-robot

/remove-lifecycle rotten

rexagod avatar Mar 19 '25 23:03 rexagod

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

k8s-ci-robot avatar Jun 17 '25 21:06 k8s-ci-robot

The Kubernetes project currently lacks enough contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

  • Mark this PR as fresh with /remove-lifecycle stale
  • Close this PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot avatar Sep 15 '25 21:09 k8s-triage-robot

The Kubernetes project currently lacks enough active contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

  • Mark this PR as fresh with /remove-lifecycle rotten
  • Close this PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

k8s-triage-robot avatar Oct 15 '25 22:10 k8s-triage-robot

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

  • Reopen this PR with /reopen
  • Mark this PR as fresh with /remove-lifecycle rotten
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

k8s-triage-robot avatar Nov 14 '25 23:11 k8s-triage-robot

@k8s-triage-robot: Closed this PR.

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

  • Reopen this PR with /reopen
  • Mark this PR as fresh with /remove-lifecycle rotten
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

k8s-ci-robot avatar Nov 14 '25 23:11 k8s-ci-robot