rename "node-role.kubernetes.io/master"

[neolit123]

KEP: http://git.k8s.io/enhancements/keps/sig-cluster-lifecycle/kubeadm/2067-rename-master-label-taint/README.md https://github.com/kubernetes/enhancements/issues/2067

## Design Details

The process will be broken into multiple stages:
- First - 1.20
- Second - Minimum deprecation period for GA features is 1 year.
Estimated 1.24, but may depend on user feedback.
- Third - one release after Second
- Fourth - one release after Third

### Renaming the "node-role.kubernetes.io/master" Node label

First stage:
- Introduce the "node-role.kubernetes.io/control-plane" label in parallel to
the "master" label.
- Announce to users that they should adapt to use the new label.

Second stage:
- Remove the "master" label and announce it to the users.

### Renaming the "node-role.kubernetes.io/master" Node taint

First stage:
- Introduce the "node-role.kubernetes.io/control-plane:NoSchedule" toleration
in the CoreDNS Deployment of kubeadm.
- Announce to users that they should do that same for their workloads.

Second stage:
- Add the "node-role.kubernetes.io/control-plane:NoSchedule" taint to Nodes.

Third stage:
- Remove the "node-role.kubernetes.io/master:NoSchedule" taint from Nodes.

Fourth stage:
- Remove the "node-role.kubernetes.io/master:NoSchedule" toleration in the CoreDNS
Deployment of kubeadm
- Announce to users that they should remove tolerations for the "master" taint in
their workloads.

---

1.20

• [x] KEP draft: https://docs.google.com/document/d/1kPs_zFconMU83LCOfBr_xKXfghn3y9VLLrTUSlCAf4s/edit#
• [x] write KEP / create issue in k/enhancements http://git.k8s.io/enhancements/keps/sig-cluster-lifecycle/kubeadm/2067-rename-master-label-taint/README.md https://github.com/kubernetes/enhancements/issues/2067
• [x] implement changes for 1.20 (deprecate the master label / taint): https://github.com/kubernetes/kubernetes/pull/95382
• [x] docs update: https://github.com/kubernetes/website/pull/24907
• [x] ADR for "master"->"control-plane": https://github.com/kubernetes/community/pull/5243

---

1.24

• [x] Update KEP https://github.com/kubernetes/enhancements/pull/3119
• [x] kind / kinder PRs: https://github.com/kubernetes/kubeadm/pull/2638 https://github.com/kubernetes-sigs/kind/pull/2595
• [x] k/k PRs https://github.com/kubernetes/kubernetes/pull/107533 https://github.com/kubernetes/kubernetes/pull/108170 https://github.com/kubernetes/kubernetes/pull/108336
• [x] k/website PRs https://github.com/kubernetes/website/pull/31391
• [x] fix retaint bug: https://github.com/kubernetes/kubernetes/pull/109841

---

1.25

• [x] fix retaint bug: https://github.com/kubernetes/kubernetes/pull/109840
• [x] update KEP: https://github.com/kubernetes/enhancements/pull/3313
• [x] k/k cleanup PRs: https://github.com/kubernetes/kubernetes/pull/110095 https://github.com/kubernetes/kubernetes/pull/110111 https://github.com/kubernetes/kubernetes/pull/110137
• [x] patch k-sigs/kind: https://github.com/kubernetes-sigs/kind/pull/2777
• [x] patch k/website (e.g. this and other places?) https://github.com/kubernetes/website/pull/33834 https://github.com/kubernetes/website/issues/33835

1.26

• [x] KEP update: https://github.com/kubernetes/enhancements/pull/3477
• [ ] cleanup k/k https://github.com/kubernetes/kubernetes/pull/112008 TODO

old issue description text:

in the kubeadm meeting of 24.06.2020 we discussed the rename of the kubeadm "master" taint/label: https://docs.google.com/document/d/1ONcoy8bOw8SWPUwXxnKeRZST3lnUxYpUv4Y6466h9Ek/edit# https://youtu.be/tycqyzrkUr8

a new k8s working group is being created to oversee the removal of offensive language in k8s. kubeadm as part of the ecosystem must comply: https://github.com/kubernetes/community/pull/4884

remaining usage of "master" and other offensive language in kubeadm:

• only usage of "master" can be found in a few places
• there is no other offensive language, such as "slave", "white/black list", or at least we are not aware of any (depends what we end up disallowing too)
• in the past, we did a big purge of "master" from code and k8s.io website docs, and replaced it with "control-plane", but a few usages remain in the code.
• kubeadm labels a control-plane Node with "node-role.kubernetes.io/master".
• kubeadm taints a control-plane Node with key "node-role.kubernetes.io/master" and effect "NoSchedule".
• kubeadm adds a toleration for its CoreDNS Deployment for the "node-role.kubernetes.io/master" taint.
• kubeadm code comments still call the label/taint as "the master label/taint".

timeline (TO BE DEFINED):

• likely no actions for 1.19.
• likely a GA deprecation policy will take action, which means that the removal will be executed over 1 year (or 3 releases).
• the exact order of tasks and how the replacement will happen is TBD, but we have ideas of how to minimize the breakage.
• a kubeadm level KEP will be required that will likely merge as "implementable" for 1.20.
• once the KEP is in place we will start executing on the tasks and notifying consumers on all possible channels.

replacement name (TO BE DEFINED):

• we have at least a couple of proposals, but we are leaning towards "Control Plane" naming.

action items before the work starts:

• [x] ACTION ITEM(@neolit123): join the SIG Cluster Lifecycle meeting and discuss:
  • usage of offensive language in SIG projects - all sub-projects have been notified.
  • discuss naming (control-plane vs ???) - no objections to control-plane.
  • discuss timeline for kubeadm and other projects - will vary between projects. still depends on the WG Naming demands, but we are hoping for a GA deprecation policy.
• [x] ACTION ITEM(@neolit123): discuss the following with WG naming:
  • usages of offensive language in kubeadm (+other SIG projects?)
  • preferred timeline?
  • naming?
  • communication channels for user complains?

---

please subscribe to this ticket for updates.

[sftim]

How about adding the new label to https://kubernetes.io/docs/reference/kubernetes-api/labels-annotations-taints/ ?

[neolit123]

not long ago these labels were made deployer specific and we cannot add them as part of the core API docs: https://github.com/kubernetes/enhancements/blob/143c03f0af52c47eab48ec54144dab6bcc86c98d/keps/sig-architecture/2019-07-16-node-role-label-use.md#use-of-node-rolekubernetesio-labels

[sftim]

The website documents more than the core API, and the point of the language seems like a good point to reconsider what's documented.

As I understand it, that KEP strongly advocated for writing sample manifests and other documentation that relies on these labels being used. I'd expect that documenting the existence of that label, with caveats, could be OK. Anyway, I'm happy to revisit this after this issue at hand gets fixed.

[justaugustus]

/wg naming

[BenTheElder]

Concrete proposal here? https://groups.google.com/d/msgid/kubernetes-sig-cluster-lifecycle/CAGDbWi9_2m3GPE3GTkbwU-qsaeV7XvzhUaHP%3DC0Guvk3zM5OFg%40mail.gmail.com

What's next for this?

[neolit123]

after the recent discussions at wg-naming i saw no objections around allowing maintainers to follow the established k8s deprecation policies for similar changes. thus, i think we can proceed with the KEP for 1.20 as outlined in the OP here.

[sftim]

In the KEP that follows from this, I would emphasize when explaining the motivation Kubernetes uses the term “control plane” (note: no dash when used in running text) to have pretty much the same meaning. Plus of course the word “master” in this context implies “slave” is linked directly to human rights abuses.

node-role.kubernetes.io/control-plane feels like the obvious replacement.

[neolit123]

NOTE: KEP draft is here: https://docs.google.com/document/d/1kPs_zFconMU83LCOfBr_xKXfghn3y9VLLrTUSlCAf4s/edit#

[BenTheElder]

Thank you for writing this up! +1 This looks plenty detailed enough to move to a KEP PR when you're ready.

On Tue, Sep 29, 2020 at 11:27 AM Lubomir I. Ivanov [email protected] wrote:

NOTE: KEP draft is here:

https://docs.google.com/document/d/1kPs_zFconMU83LCOfBr_xKXfghn3y9VLLrTUSlCAf4s/edit#

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/kubernetes/kubeadm/issues/2200#issuecomment-700898607, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAHADK6U6JN2EKQGGL7OIEDSIIRKFANCNFSM4OIMHSOA .

[neolit123]

In the KEP that follows from this, I would emphasize when explaining the motivation Kubernetes uses the term “control plane” (note: no dash when used in running text) to have pretty much the same meaning. Plus of course the word “master” in this context implies “slave” is linked directly to human rights abuses.

@sftim "master" actually is not the same as "control plane". historically "master" in the k8s project meant:

• a Node that hosts control plane components
• an instance of the kube-apiserver
• EDIT: i guess it also has another meaning - admin.

in kubeadm, we have usage of the first case and a rename to control-plane has a clear intent.

i don't think we should include details around that or topics like "human rights" in the kubeadm KEP, so i'm defering this to the WG Naming group.

[qnetter]

Is there prior art in any distributor tooling? I know Rancher's RKE uses etcd, control-plane, and worker node naming.

[neolit123]

Is there prior art in any distributor tooling?

usage of the key node-role.kubernetes.io/master can be found in both commercial (closed source) and open-source projects.

[neolit123]

"enhancement" tracking issue: https://github.com/kubernetes/enhancements/issues/2067 KEP PR: https://github.com/kubernetes/enhancements/pull/2068

the plan for 1.20 is clear. for the later stages the feedback on the draft is now part of the KEP PR, but we can amend the proposal if needed.

[neolit123]

the proposal merged: http://git.k8s.io/enhancements/keps/sig-cluster-lifecycle/kubeadm/2067-rename-master-label-taint/README.md 1.20 is the first stage of the rename. the plan for the later stages may change depending on feedback.

tracking issue at k/e: https://github.com/kubernetes/enhancements/issues/2067

[neolit123]

PR for the 1.20 changes in kubeadm source code: https://github.com/kubernetes/kubernetes/pull/95382

[neolit123]

PR for the 1.20 changes in kubeadm source code: kubernetes/kubernetes#95382

this is merging.

[celestehorgan]

@neolit123 I believe all your sub-items on this task are done. Is this issue ready to be closed? :)

[neolit123]

@celestehorgan hi, we should keep this open until the transition from master -> control-plane has fully completed in future releases. if you'd like we can be removed the WG Naming label via /remove-wg naming or just remove it from the WG Naming GitHub project tracking?

[fejta-bot]

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community. /lifecycle stale

[neolit123]

/remove-lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[sftim]

Would it be OK to mark this one as frozen?

[neolit123]

There are action items for 1.24.

/remove-lifecycle stale

[calvin0327]

1.23 will be released soon, can we delete the corresponding logic in 1.24?

[neolit123]

I will assign myself to work on this in 1.24. Thanks. On Dec 2, 2021 12:03, "Calvin Chen" @.***> wrote:

1.23 will be released soon, can we delete the corresponding logic in 1.24?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/kubernetes/kubeadm/issues/2200#issuecomment-984475670, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACRATHNANYR47LNQWP2XCTUO5AA7ANCNFSM4OIMHSOA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[neolit123]

WIP PR for 1.24 (stage 2) is here: https://github.com/kubernetes/kubernetes/pull/107533

[neolit123]

we need to send a fix for the e2e_kubeadm suite in the release-1.23 branch of k/k: https://github.com/kubernetes/kubernetes/pull/108128#issuecomment-1041822415 as this broke the kubeadm CP skew tests.

i will do that later today, probably.

[perosb]

just upgraded to 1.24. shouldn't the control-plane taint only be added to nodes that have a master taint if this is a "rename"? I had removed master taint from control plane nodes which suddenly became unschedulable.

[neolit123]

that's a good point.. unfortunately that ended as an oversight on our side in this code

you are going to have to remove the new taint again.

[perosb]

that's a good point.. unfortunately that ended as an oversight on our side in this code

you are going to have to remove the new taint again.

do I read it correct that this will get added for all future upgrades too? as long as the code is there, at least for all 1.24.x? the master taint wasn't previously added during every upgrade.