enhancements icon indicating copy to clipboard operation
enhancements copied to clipboard

Allow informers for getting a stream of data instead of chunking

Open p0lyn0mial opened this issue 3 years ago • 111 comments

Enhancement Description

  • One-line enhancement description (can be used as a release note): informers can get a stream of data to prime the caches
  • Kubernetes Enhancement Proposal: https://github.com/kubernetes/enhancements/blob/master/keps/sig-api-machinery/3157-watch-list/README.md
  • Discussion Link:
  • Primary contact (assignee): @p0lyn0mial
  • Responsible SIGs: sig-apimachinery, sig-scalability
  • Enhancement target (which target equals to which milestone):
    • Alpha release target (x.y): 1.27
    • Beta release target (x.y): 1.32
    • Stable release target (x.y): TBD
  • [X] Alpha
    • [X] KEP (k/enhancements) update PR(s):
      • https://github.com/kubernetes/enhancements/pull/3142
      • https://github.com/kubernetes/enhancements/pull/3822
    • [X] Code (k/k) update PR(s):
      • https://github.com/kubernetes/kubernetes/pull/106477
      • https://github.com/kubernetes/kubernetes/pull/117277
      • https://github.com/kubernetes/kubernetes/pull/117410
    • [X] Docs (k/website) update PR(s):
      • https://github.com/kubernetes/website/pull/39993
  • [ ] Beta
    • [x] KEP (k/enhancements) update PR(s):
      • [x] 1.30 https://github.com/kubernetes/enhancements/pull/4453
      • [x] 1.32 https://github.com/kubernetes/enhancements/pull/4836
    • [ ] Code (k/k) update PR(s):
      • [x] 1.30 https://github.com/kubernetes/kubernetes/pull/122830
      • [x] 1.31 https://github.com/kubernetes/kubernetes/pull/122791
      • [ ]
    • [ ] Docs (k/website) update(s):
      • [ ] https://github.com/kubernetes/website/pull/45283
      • [ ] https://github.com/kubernetes/website/pull/45285
      • [ ] https://github.com/kubernetes/website/pull/45286
      • [ ] https://github.com/kubernetes/website/pull/47261

Please keep this description up to date. This will help the Enhancement Team to track the evolution of the enhancement efficiently.

/sig api-machinery /sig scalability

p0lyn0mial avatar Jan 18 '22 12:01 p0lyn0mial

Hi @p0lyn0mial ! 1.24 Enhancements team here. Just checking in as we approach enhancements freeze on 18:00pm PT on Thursday Feb 3rd. This enhancements is targeting alpha for 1.24 Here’s where this enhancement currently stands:

  • [ ] Updated KEP file using the latest template has been merged into the k/enhancements repo. #3142
  • [x] KEP status is marked as implementable for this release with latest-milestone: 1.24
  • [x] KEP has a test plan section filled out. ~~- [ ] KEP has up to date graduation criteria.~~ -- not needed for alpha
  • [ ] KEP has a production readiness review that has been completed and merged into k/enhancements -- PRR not found

The status of this enhancement is track as at risk Thanks!

gracenng avatar Jan 20 '22 15:01 gracenng

Hi @gracenng! Many thanks for the heads-up. Could you please double-check the last point? The KEP has the Production Readiness Review Questionnaire. In addition to that, I have filled the required bits for the alpha version - it just hasn't been reviewed yet.

p0lyn0mial avatar Jan 20 '22 15:01 p0lyn0mial

Hi @p0lyn0mial, Thanks for the reply. Your PRR Questionaire in the KEP needs to be approved by the PRR team, which will result in a file in https://github.com/kubernetes/enhancements/tree/master/keps/prod-readiness/ Your sig has not made a folder there yet, please check out the format.

Regarding your second point, I am tracking your KEP files and the first bullet will be checked when the PR is merged. Does that answer your question?

gracenng avatar Jan 22 '22 02:01 gracenng

Hi @p0lyn0mial , 1.24 Enhancements Team here.

Reaching out as we're less than a week away from Enhancement Freeze on Thursday, February 3rd. There's no update for this enhancement since last checkin, let me know if I missed anything. Current status is at risk

gracenng avatar Jan 30 '22 09:01 gracenng

Hi @gracenng, it looks like it won't make it to 1.24. Please leave me some instructions on how to update the KEP so it won't show up on your query.

p0lyn0mial avatar Jan 31 '22 08:01 p0lyn0mial

No worries I’ll take care of it

On Mon, Jan 31, 2022 at 12:42 AM Lukasz Szaszkiewicz < @.***> wrote:

Hi @gracenng https://github.com/gracenng, it looks like it won't make it to 1.24. Please leave me some instructions on how to update the KEP so it won't show up on your query.

— Reply to this email directly, view it on GitHub https://github.com/kubernetes/enhancements/issues/3157#issuecomment-1025498783, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKCRLOZTCK54ONCDRDEXZCTUYZDP3ANCNFSM5MHDCLRQ . You are receiving this because you were mentioned.Message ID: @.***>

gracenng avatar Jan 31 '22 16:01 gracenng

@p0lyn0mial how is feature going? we met a lot of server overloading scene(kube-apiserver oom or master oom), looking forward to this feature. if you need help,you can assign some tasks to me.

Dingshujie avatar Apr 18 '22 02:04 Dingshujie

@Dingshujie BTW, if you have more than a few hundred nodes, then this fix will probably help to reduce API server load https://github.com/kubernetes/kubernetes/pull/109381 . (If you have more than a few thousand nodes it will definitely help). It solves a bug where the kubelets would keep doing the same listing calls over and over again (typically immediately after a restart of the API server). Obvoiusly, other list issues are possible too, so that fix won't solve them all for you. But it might help.

JohnRusk avatar May 26 '22 06:05 JohnRusk

@Dingshujie BTW, if you have more than a few hundred nodes, then this fix will probably help to reduce API server load kubernetes/kubernetes#109381 . (If you have more than a few thousand nodes it will definitely help). It solves a bug where the kubelets would keep doing the same listing calls over and over again (typically immediately after a restart of the API server). Obvoiusly, other list issues are possible too, so that fix won't solve them all for you. But it might help.

@JohnRusk thanks, i already cherry-pick this to 1.23 branch. but this fixs can't slove the situation that custom-defined operator list issues. and finally expect all list from watch cache

Dingshujie avatar May 27 '22 05:05 Dingshujie

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue or PR as fresh with /remove-lifecycle stale
  • Mark this issue or PR as rotten with /lifecycle rotten
  • Close this issue or PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot avatar Aug 25 '22 05:08 k8s-triage-robot

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue or PR as fresh with /remove-lifecycle rotten
  • Close this issue or PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

k8s-triage-robot avatar Sep 24 '22 06:09 k8s-triage-robot

/remove-lifecycle rotten

wojtek-t avatar Sep 26 '22 06:09 wojtek-t

/milestone v1.26

p0lyn0mial avatar Sep 28 '22 12:09 p0lyn0mial

@p0lyn0mial: You must be a member of the kubernetes/milestone-maintainers GitHub team to set the milestone. If you believe you should be able to issue the /milestone command, please contact your Milestone Maintainers Team and have them propose you as an additional delegate for this responsibility.

In response to this:

/milestone v1.26

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

k8s-ci-robot avatar Sep 28 '22 12:09 k8s-ci-robot

/stage alpha

p0lyn0mial avatar Sep 28 '22 13:09 p0lyn0mial

/label lead-opted-in

p0lyn0mial avatar Sep 28 '22 13:09 p0lyn0mial

/milestone v1.26

deads2k avatar Sep 28 '22 13:09 deads2k

@p0lyn0mial: Can not set label lead-opted-in: Must be member in one of these teams: [release-team-enhancements sig-api-machinery-leads sig-apps-leads sig-architecture-leads sig-auth-leads sig-autoscaling-leads sig-cli-leads sig-cloud-provider-leads sig-cluster-lifecycle-leads sig-contributor-experience-leads sig-docs-leads sig-instrumentation-leads sig-k8s-infra-leads sig-multicluster-leads sig-network-leads sig-node-leads sig-release-leads sig-scalability-leads sig-scheduling-leads sig-security-leads sig-storage-leads sig-testing-leads sig-windows-leads]

In response to this:

/label lead-opted-in

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

k8s-ci-robot avatar Sep 28 '22 13:09 k8s-ci-robot

/label lead-opted-in /label tracked/yes

deads2k avatar Sep 28 '22 13:09 deads2k

@deads2k: Can not set label tracked/yes: Must be member in one of these teams: [release-team-enhancements release-team-leads]

In response to this:

/label lead-opted-in /label tracked/yes

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

k8s-ci-robot avatar Sep 28 '22 13:09 k8s-ci-robot

@gracenng please tag this issue with /label tracked/yes so that it shows up on the https://github.com/orgs/kubernetes/projects/98/views/1

p0lyn0mial avatar Sep 28 '22 13:09 p0lyn0mial

/label tracked/yes

marosset avatar Sep 28 '22 21:09 marosset

Hello @p0lyn0mial 👋, 1.26 Enhancements team here.

Just checking in as we approach enhancements freeze on 18:00 PDT on Thursday 6th October 2022.

This enhancement is targeting for stage alpha for 1.26 (correct me, if otherwise)

Here's where this enhancement currently stands:

  • [ ] KEP readme using the latest template has been merged into the k/enhancements repo.
  • [ ] KEP status is marked as implementable for latest-milestone: 1.26
  • [ ] KEP readme has a updated detailed test plan section filled out
  • [ ] KEP readme has up to date graduation criteria
  • [ ] KEP has a production readiness review that has been completed and merged into k/enhancements.

For this KEP, please make sure https://github.com/kubernetes/enhancements/pull/3561/ includes updates that address all of the above requirements.

The status of this enhancement is marked as at risk. Please keep the issue description up-to-date with appropriate stages as well. Thank you!

marosset avatar Sep 28 '22 21:09 marosset

Hello @p0lyn0mial 👋, 1.26 Enhancements team here.

Just checking in as we approach enhancements freeze on 18:00 PDT on Thursday 6th October 2022.

This enhancement is targeting for stage alpha for 1.26 (correct me, if otherwise)

Here's where this enhancement currently stands:

  • [ ] KEP readme using the latest template has been merged into the k/enhancements repo.
  • [x] KEP status is marked as implementable for latest-milestone: 1.26
  • [ ] KEP readme has a updated detailed test plan section filled out
  • [ ] KEP readme has up to date graduation criteria
  • [x] KEP has a production readiness review that has been completed and merged into k/enhancements.

For this KEP, we would just need to update the following:

  • Add graduation to https://github.com/kubernetes/enhancements/tree/master/keps/sig-api-machinery/3157-watch-list#graduation-criteria
  • Update https://github.com/kubernetes/enhancements/tree/master/keps/sig-api-machinery/3157-watch-list#test-plan - to match the latest kep template

The status of this enhancement is marked as at risk. Please keep the issue description up-to-date with appropriate stages as well. Thank you!

marosset avatar Oct 05 '22 21:10 marosset

Hello 👋, 1.26 Enhancements Lead here.

Unfortunately, this enhancement did not meet requirements for enhancements freeze.

If you still wish to progress this enhancement in v1.26, please file an exception request. Thanks!

/milestone clear /label tracked/no /remove-label tracked/yes /remove-label lead-opted-in

rhockenbury avatar Oct 07 '22 01:10 rhockenbury

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue or PR as fresh with /remove-lifecycle stale
  • Mark this issue or PR as rotten with /lifecycle rotten
  • Close this issue or PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot avatar Jan 05 '23 02:01 k8s-triage-robot

/label lead-opted-in /label tracked/yes /remove-lifecycle stale

wojtek-t avatar Jan 23 '23 11:01 wojtek-t

@wojtek-t: Can not set label tracked/yes: Must be member in one of these teams: [release-team-enhancements release-team-leads]

In response to this:

/label lead-opted-in /label tracked/yes /remove-lifecycle stale

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

k8s-ci-robot avatar Jan 23 '23 11:01 k8s-ci-robot

/milestone v1.26 /label lead-opted-in

deads2k avatar Jan 23 '23 23:01 deads2k

/milestone v1.27 /label lead-opted-in

deads2k avatar Jan 23 '23 23:01 deads2k