Dynamic Kubelet Configuration

[mtaufen]

Dynamic Kubelet Configuration

• One-line feature description (can be used as a release note): New Kubelet configurations can be rolled out in a live cluster
• Primary contact (assignee): @mtaufen
• Responsible SIGs: sig-node
• Design proposal (community repo)
• Documentation:
  • Low-level manual workflow
• Tests:
  • https://k8s-testgrid.appspot.com/sig-node-kubelet#kubelet-serial-gce-e2e&include-filter-by-regex=DynamicKubeletConfig
• Reviewer(s) - (for LGTM) recommend having 2+ reviewers (at least one from code-area OWNERS file) agreed to review. Reviewers from multiple companies preferred: @liggitt @thockin @dchen1107 @dashpole
• Approver (likely from SIG/area to which feature belongs): @dchen1107
• Feature targets (which target equals to which milestone):
  • Alpha release target (1.8):
    • [x] Foundational primitives and Kubelet behavior implemented behind an Alpha feature gate
  • 1.9:
    • [x] Incremental improvements working towards 1.10 goals
  • 1.10:
    • [x] Improved testing, see https://github.com/kubernetes/kubernetes/issues/50217
    • [x] Kubelet configuration type reviewed and moved to beta
    • [x] Kubelet loading configuration from a file moved to beta
  • Beta release target (1.11):
    • [x] Node authorizer automatically allows access to the source specified by Node.Spec.ConfigSource https://github.com/kubernetes/kubernetes/pull/60100
    • [x] Unpack remote configs to directories on the node: https://github.com/kubernetes/kubernetes/pull/59692
    • [x] Make the Kubelet config key explicit:
      • [x] PR: https://github.com/kubernetes/kubernetes/pull/59847
    • [x] Simplify skew scenarios/debugging by respecting ConfigMap mutations
      • [x] Issue: https://github.com/kubernetes/kubernetes/issues/61643
      • [x] PR: https://github.com/kubernetes/kubernetes/pull/63221
    • [x] Expose a structured status for dynamic config, rather than a node condition:
      • [x] Issue: https://github.com/kubernetes/kubernetes/issues/56896
      • [x] PR: https://github.com/kubernetes/kubernetes/pull/63314
    • [x] Expose metrics for Kubelet config:
      • [x] PR: https://github.com/kubernetes/kubernetes/pull/57527
    • [x] Kubelet config validation should respect target feature gate combination
      • [x] Issue: https://github.com/kubernetes/kubernetes/issues/63305
      • [x] PR: https://github.com/kubernetes/kubernetes/pull/63409
    • [x] Improve test coverage of the file utils (pkg/kubelet/kubeletconfig/util/files/files.go):
      • [x] PR: https://github.com/kubernetes/kubernetes/pull/63496
    • [x] Document the structure of the file tree under --dynamic-config-dir, other docs updates for 1.11
      • [x] PR: https://github.com/kubernetes/website/pull/8766
    • [x] Document per-field advise for cluster administrators and k8s-as-a-service providers
      • [x] PR: https://github.com/kubernetes/kubernetes/pull/64354
    • [x] Dynamic kubelet configuration moved to beta

[mtaufen]

Alpha PR: https://github.com/kubernetes/kubernetes/pull/46254

[dchen1107]

Punt this to v1.8.

[luxas]

@mtaufen Any user/admin-facing docs ready for this feature?

[mtaufen]

No, I really need to write something to teach people how to play with the alpha. What's the preferred place for these docs again?

[luxas]

@mtaufen https://kubernetes.io/docs (https://github.com/kubernetes/kubernetes.github.io) I'll definitely read what it came to look like finally

[mtaufen]

@luxas https://github.com/kubernetes/kubernetes.github.io/pull/5523

[idvoretskyi]

@mtaufen can you update the features tracking board with the relevant data?

Thanks.

[mtaufen]

Done

[pipejakob]

Looks like the proposal has moved to:

https://github.com/kubernetes/community/blob/master/contributors/design-proposals/node/dynamic-kubelet-configuration.md

@mtaufen Want to update the link in the description?

[mtaufen]

Done, thanks!

[luxas]

Updated the labels here, @mtaufen please share the beta graduation requirements here for visibility

[mtaufen]

Deprecated and experimental fields removed from the KubeletConfiguration type in https://github.com/kubernetes/kubernetes/pull/53088. Next steps: push the type to beta, chase down loose ends (testing, node authorizer, etc.) and push dynamic config to beta.

[fejta-bot]

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or @fejta. /lifecycle stale

[mtaufen]

/lifecycle frozen

[idvoretskyi]

@mtaufen can you comment on the actual feature status?

[mtaufen]

still pushing for beta in 1.10

[idvoretskyi]

Ref https://github.com/kubernetes/kubernetes/pull/55718

[timothysc]

From https://github.com/kubernetes/kubernetes/pull/60412 it appears beta is a push to 1.11

[mtaufen]

@timothysc that is correct. I was hoping the node authorizer checkbox for v1.10 could at least make it, but that seems increasingly unlikely given the time constraints...

[luxas]

This is then wrongly flagged beta in http://blog.kubernetes.io/2018/03/first-beta-version-of-kubernetes-1-10.html, as this label is beta here, swapping. cc @nickchase (thanks for all your hard work on the article, seems to have been a communication issue here)

[mtaufen]

Thanks for catching that @luxas. @nickchase what do we need to do to update the post?

[justaugustus]

@mtaufen Any plans for this in 1.11?

If so, can you please ensure the feature is up-to-date with the appropriate:

• Description
• Milestone
• Assignee(s)
• Labels:
  • stage/{alpha,beta,stable}
  • sig/*
  • kind/feature

cc @idvoretskyi

[mtaufen]

@justaugustus does stage indicate the target for the milestone, or the current status?

[justaugustus]

@mtaufen the stage represents the target for the current milestone (v1.11).

[mtaufen]

Thanks, updated.

[mdlinville]

@mtaufen please fill out the appropriate line item of the 1.11 feature tracking spreadsheet and open a placeholder docs PR against the release-1.11 branch by 5/25/2018 (tomorrow as I write this) if new docs or docs changes are needed and a relevant PR has not yet been opened.

[mtaufen]

@mistyhacks done, thanks!

[kacole2]

@mtaufen This feature was worked on in the previous milestone, so we'd like to check in and see if there are any plans for this to graduate stages in Kubernetes 1.12 since there is nothing in the original post. This still has the 1.11 milestone as well so we need to update it accordingly.

If there are any updates, please explicitly ping @justaugustus, @kacole2, @robertsandoval, @rajendar38 to note that it is ready to be included in the Features Tracking Spreadsheet for Kubernetes 1.12.

---

Please note that the Features Freeze is July 31st, after which any incomplete Feature issues will require an Exception request to be accepted into the milestone.

In addition, please be aware of the following relevant deadlines:

• Docs deadline (open placeholder PRs): 8/21
• Test case freeze: 8/28

Please make sure all PRs for features have relevant release notes included as well.

Happy shipping!

[mtaufen]

No plans to graduate for 1.12.

[kacole2]

Hi This enhancement has been tracked before, so we'd like to check in and see if there are any plans for this to graduate stages in Kubernetes 1.13. This release is targeted to be more ‘stable’ and will have an aggressive timeline. Please only include this enhancement if there is a high level of confidence it will meet the following deadlines:

• Docs (open placeholder PRs): 11/8
• Code Slush: 11/9
• Code Freeze Begins: 11/15
• Docs Complete and Reviewed: 11/27

Please take a moment to update the milestones on your original post for future tracking and ping @kacole2 if it needs to be included in the 1.13 Enhancements Tracking Sheet

Thanks!