🐛 prevent leader election when shutting down a non-elected manager

[alexandremahdhaoui]

Context

This PR should address a bug where the runnable group for Leader Election would start while shutting down the manager: https://github.com/kubernetes-sigs/controller-runtime/issues/2719

What's happening?

• https://github.com/kubernetes-sigs/controller-runtime/blob/395cfc7486e652d19fe1b544a436f9852ba26e4f/pkg/manager/internal.go#L344
• https://github.com/kubernetes-sigs/controller-runtime/blob/395cfc7486e652d19fe1b544a436f9852ba26e4f/pkg/manager/internal.go#L521
• https://github.com/kubernetes-sigs/controller-runtime/blob/395cfc7486e652d19fe1b544a436f9852ba26e4f/pkg/manager/internal.go#L521
• Causes the side-effect of starting the runnableGroup: https://github.com/kubernetes-sigs/controller-runtime/blob/395cfc7486e652d19fe1b544a436f9852ba26e4f/pkg/manager/runnable_group.go#L277

However when leader election is enabled, a non-leader manager will never start the LeaderElection runnable group. Thus, the sync.Once allow starting a new election during shutdown. The change in this PR ensures Start is ineffective during shutdown.

• This func calls the manager.LeaderElection's runnableGroup.Start() https://github.com/kubernetes-sigs/controller-runtime/blob/395cfc7486e652d19fe1b544a436f9852ba26e4f/pkg/manager/internal.go#L557
• Which is obviously only called when a member becomes leader: https://github.com/kubernetes-sigs/controller-runtime/blob/395cfc7486e652d19fe1b544a436f9852ba26e4f/pkg/manager/internal.go#L568
• In the case leader election is enabled: https://github.com/kubernetes-sigs/controller-runtime/blob/395cfc7486e652d19fe1b544a436f9852ba26e4f/pkg/manager/internal.go#L429

Changes

• On shutdown, perform a noop call to the sync.Once func associated with the LeaderElection runnableGroup to prevent it from starting a new leader election.

Testing

Please note that I'd like to add tests for this change and would be very happy if anyone could give me a hint. I've looked into the tests in pkg/manager/manager_test.go and was wondering if line 1230 would be a good place to add my test case?

[linux-foundation-easycla[bot]]

The committers listed above are authorized under a signed CLA.

• :white_check_mark: login: alexandremahdhaoui / name: Alexandre Mahdhaoui (d0c20b4307e80e2767395dc317f7aa7ca29f83a4)

[k8s-ci-robot]

Welcome @alexandremahdhaoui!

It looks like this is your first PR to kubernetes-sigs/controller-runtime 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes-sigs/controller-runtime has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. :smiley:

[k8s-ci-robot]

Hi @alexandremahdhaoui. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[vincepri]

/lgtm

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: alexandremahdhaoui, alvaroaleman

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• ~~OWNERS~~ [alvaroaleman]

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

[alvaroaleman]

/cherrypick release-0.17

[k8s-infra-cherrypick-robot]

@alvaroaleman: new pull request created: #2752

In response to this:

/cherrypick release-0.17

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.