ingress-nginx icon indicating copy to clipboard operation
ingress-nginx copied to clipboard

sleep before shutdown

Open wuzhuoquan opened this issue 1 year ago • 4 comments

What this PR does / why we need it:

Types of changes

  • [x] Bug fix (non-breaking change which fixes an issue)
  • [ ] New feature (non-breaking change which adds functionality)
  • [ ] CVE Report (Scanner found CVE and adding report)
  • [ ] Breaking change (fix or feature that would cause existing functionality to change)
  • [ ] Documentation only

Which issue/s this PR fixes

How Has This Been Tested?

The ingress-controller deploy in k8s cluster,when deleting the ingress-controller pod, k8s controller will remove this pod from endpoint list, and then the container execute waitshutdown to sent SIGTERM to the nginx-ingress-controller process according the preStop setting.

After remove the pod from endpoint list, kube-proxy listwatch the change of endpoint and update the iptables rule to remove the pod IP.

But sometimes kube-proxy and iptables remote the pod and ip not as fast as executing waitshutdown, so if at this time,a new connection request may be send to the deleting pod but waitshutdown is executed,will return connection refused to client. image

So I think before sent SIGTERM to the nginx-ingress-controller process, it should sleep for a while to avoid this problem and make it more smooth.

Checklist:

  • [ ] My change requires a change to the documentation.
  • [ ] I have updated the documentation accordingly.
  • [ ] I've read the CONTRIBUTION guide
  • [ ] I have added unit and/or e2e tests to cover my changes.
  • [ ] All new and existing tests passed.

wuzhuoquan avatar Aug 13 '24 11:08 wuzhuoquan

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: wuzhuoquan Once this PR has been reviewed and has the lgtm label, please assign cpanato for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot avatar Aug 13 '24 11:08 k8s-ci-robot

This issue is currently awaiting triage.

If Ingress contributors determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

k8s-ci-robot avatar Aug 13 '24 11:08 k8s-ci-robot

Hi @wuzhuoquan. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

k8s-ci-robot avatar Aug 13 '24 11:08 k8s-ci-robot

Deploy Preview for kubernetes-ingress-nginx canceled.

Name Link
Latest commit 305421fb805170909016b347087b6d4b777a29dd
Latest deploy log https://app.netlify.com/sites/kubernetes-ingress-nginx/deploys/66bb3cb5bec6540008fc7a3c

netlify[bot] avatar Aug 13 '24 11:08 netlify[bot]

We recently had a discussion about this for a different reason (AWS NLB) and came to the conclusion that a wait, at least such a static value, is not a good way to solve this and the actual value might vary from use-case to use-case.

You can read more here: https://github.com/kubernetes/ingress-nginx/issues/11890.

I therefore close this PR and ask you to implement this on your own in the according chart value.

Gacko avatar Sep 10 '24 17:09 Gacko