ingress-nginx icon indicating copy to clipboard operation
ingress-nginx copied to clipboard

Allow tuning leader-election parameters or turn off it

Open fanminshi opened this issue 3 years ago • 2 comments

In a single node k8s setup, there is no need to enable leader-election inside the ingress-controller. The issue with constant leader election lease renewal is that it adds unwanted writes to etcd. In return, those writes can reduce disk life-span on small and cheap edge device.

By analyzes etcd PUT operation, I noticed the following PUT within 8 minutes span.

    635 PUT
    634 --
     73 /registry/configmaps/ingress-nginx/ingress-controller-leader

Specially, I would like to tune those parameters via command-line tool. https://github.com/kubernetes/ingress-nginx/blob/c85765a015ea6709d161eccd42ef6134981c04b4/internal/ingress/controller/status.go#L117-L124

Or simply turn it off via a command line flag.

Thanks,

fanminshi avatar Aug 10 '22 12:08 fanminshi

@fanminshi: This issue is currently awaiting triage.

If Ingress contributors determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

k8s-ci-robot avatar Aug 10 '22 12:08 k8s-ci-robot

thanks for your report.

I have opened a PR to use LeaseLock for election. https://github.com/kubernetes/ingress-nginx/pull/8921 It will not use configmap anymore.

tao12345666333 avatar Aug 10 '22 19:08 tao12345666333

@tao12345666333 thanks for the PR. However, the new PR will redirect writes to /registry/leases/ingress-nginx/ingress-controller-leader instead. That will not solved my problem. It will be nice to makettl := 30 * time.Second configurable over the a flag. I can put up a PR if needed.

fanminshi avatar Aug 11 '22 09:08 fanminshi

Hahaha, yes. Should there be no election when there is only 1 replica?

tao12345666333 avatar Aug 11 '22 09:08 tao12345666333

Of course, adjusting the time of ttl is also a solution

tao12345666333 avatar Aug 11 '22 09:08 tao12345666333

Should there be no election when there is only 1 replica? yes, that should be the expected behavior.

fanminshi avatar Aug 11 '22 12:08 fanminshi

I think we should add a feature flag such as leader-elect=false to turn it off. kube-scheduler has the following flag.

--leader-elect     Default: true
--
  | Start a leader election client and gain leadership before executing the main loop. Enable this when running replicated components for high availability.

https://kubernetes.io/docs/reference/command-line-tools-reference/kube-scheduler/

fanminshi avatar Aug 12 '22 13:08 fanminshi

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue or PR as fresh with /remove-lifecycle stale
  • Mark this issue or PR as rotten with /lifecycle rotten
  • Close this issue or PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot avatar Nov 10 '22 13:11 k8s-triage-robot

/remove-lifecycle stale

ensonic avatar Nov 10 '22 17:11 ensonic

@ensonic @fanminshi do you mind opening a PR for it?

Thanks!

rikatz avatar Feb 28 '24 21:02 rikatz

/help /good-first-issue

rikatz avatar Feb 28 '24 21:02 rikatz

@rikatz: This request has been marked as suitable for new contributors.

Guidelines

Please ensure that the issue body includes answers to the following questions:

  • Why are we solving this issue?
  • To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points?
  • Does this issue have zero to low barrier of entry?
  • How can the assignee reach out to you for help?

For more details on the requirements of such an issue, please see here and ensure that they are met.

If this request no longer meets these requirements, the label can be removed by commenting with the /remove-good-first-issue command.

In response to this:

/help /good-first-issue

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

k8s-ci-robot avatar Feb 28 '24 21:02 k8s-ci-robot