enhancements icon indicating copy to clipboard operation
enhancements copied to clipboard

ConfigMap / Secret Orchestration

Open kfox1111 opened this issue 6 years ago • 36 comments

This change adds a KEP for ConfigMap / Secret Orchestration support.

kfox1111 avatar Apr 10 '19 19:04 kfox1111

Hi @kfox1111. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

k8s-ci-robot avatar Apr 10 '19 19:04 k8s-ci-robot

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: kfox1111 To fully approve this pull request, please assign additional approvers. We suggest the following additional approver: mattfarina

If they are not already assigned, you can assign the PR to them by writing /assign @mattfarina in a comment when ready.

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot avatar Apr 10 '19 19:04 k8s-ci-robot

/ok-to-test

justaugustus avatar Apr 28 '19 02:04 justaugustus

@JoelSpeed

kfox1111 avatar May 21 '19 19:05 kfox1111

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale

fejta-bot avatar Aug 24 '19 19:08 fejta-bot

/remove-lifecycle stale

kfox1111 avatar Aug 26 '19 15:08 kfox1111

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale

fejta-bot avatar Nov 24 '19 16:11 fejta-bot

/remove-lifecycle stale

Bessonov avatar Nov 24 '19 22:11 Bessonov

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale

fejta-bot avatar Feb 22 '20 22:02 fejta-bot

/remove-lifecycle stale

Yeah, everyone should be notified that no body cares about this issue for the last 90d.

Bessonov avatar Feb 23 '20 21:02 Bessonov

@Bessonov We need a broken heart reaction.

:broken_heart:

kfox1111 avatar Feb 24 '20 16:02 kfox1111

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale

fejta-bot avatar Jun 07 '20 17:06 fejta-bot

/remove-lifecycle stale

Bessonov avatar Jun 07 '20 17:06 Bessonov

I don't know why, but github is letting me respond to some comments and not others... responding to the rest here:

Why does watch require snapshot to be true? Also, wouldn't it make more sense to opt into this on workload-level (Deployment, DS, Statefulset) because if immutability is wanted, we already have the immutable configmap/secret fature?

The way watch is working in this context, without a snapshot a watch wouldn't have anything to trigger off of in the described implementation if there wasn't a snapshotted new configmap name to push into the new replicaset.

It can not be made to work on pods, pod volumes are immutable. I would also argue it should not be made on pods, because that doesn't allow to have any control in case the new revision causes issue.

I think more or less the same is true for Jobs, you should use immutable configmaps/secrets for them to get deterministic results

I agree on pods and jobs. Thats why it was stated it only works on deployments, statefulsets and daemonsets. Only those three objects have a concept of "version" of the podtemplate where you can roll forwards and backwards. Its these "versions" that need immutable configmaps/secrets during the life of that "version". The simplest thing on the user I can think of is to version the configmap at the same time the podtempate is versioned and keep the lifecycle the same. ReplicaSet and configmaps get created at the same time, and get deleted at the same time.

Maybe an example will help. User uploads configmap foo and then deployment foo. its set as watched and snapshotted. when the deployment is created, configmap foo is copied to immutable configmap foo-1 and replicaset foo-1 is created pointing at configmap foo-1 in the podtempate

User then edits configmap foo deployment notices it, copies configmap foo to immutable configmap foo-2 and creates replicaset foo-2 pointing at configmap foo-2.

All the pods then in replicaset foo-1 are always consistent, and all the pods in replicaset foo-2 are always consistent. You can roll forwards and back between foo-1 and foo-2 and it always works consistently. This doesn't work consistently today unless you are very careful and add a lot of manual, error prone steps.

Then if the user deletes replicaset foo-1 cause they are done with it (or the system does for them), configmap foo-1 gets garbage collected too. If the user deletes the deployment, also all the snapshotted configmaps associated with the deployment go away too.

So, for the user, the cognitive burden is just, one configap with their config, and one deployment for orchestrating the app. Rolling forward/back just works. That is how they think it works when they first go into Kubernetes and then find out its much more complected today then that.

kfox1111 avatar Jul 17 '20 16:07 kfox1111

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale

fejta-bot avatar Jan 13 '21 17:01 fejta-bot

For posterity, one workaround for "I want my pod to rotate when config changes" is to put the config in a pod annotation and then use the downward api to get the content of that annotation into an env var or a file. I am very certain this was never intended to be used like that, but its the least bad workaround I am aware of.

alvaroaleman avatar Jan 13 '21 17:01 alvaroaleman

Wouldn’t it then be simpler to put the config into an environment variable?

On Wed, Jan 13, 2021 at 10:20 Alvaro Aleman [email protected] wrote:

For posterity, one workaround for "I want my pod to rotate when config changes" is to put the config in a pod annotation and then use the downward api to get the content of that annotation into an env var or a file. I am very certain this was never intended to be used like that, but its the least bad workaround I am ware of.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/kubernetes/enhancements/pull/948#issuecomment-759596487, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABB6TFGEBIKCOII3Y4TUDVLSZXI5PANCNFSM4HFAHQ5Q .

--

Dharma Bellamkonda

dharmab avatar Jan 13 '21 18:01 dharmab

Wouldn’t it then be simpler to put the config into an environment variable?

Yeah, but many applications do not support reading their config from an env var, the downward api also allows to get it into a file

alvaroaleman avatar Jan 13 '21 18:01 alvaroaleman

Stale issues rot after 30d of inactivity. Mark the issue as fresh with /remove-lifecycle rotten. Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community. /lifecycle rotten

fejta-bot avatar Feb 12 '21 18:02 fejta-bot

Still a problem.

/remove-lifecycle rotten

kfox1111 avatar Feb 15 '21 18:02 kfox1111

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community. /lifecycle stale

fejta-bot avatar May 16 '21 18:05 fejta-bot

Still a problem.

/remove-lifecycle rotten

kfox1111 avatar May 17 '21 16:05 kfox1111

Stale issues rot after 30d of inactivity. Mark the issue as fresh with /remove-lifecycle rotten. Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community. /lifecycle rotten

fejta-bot avatar Jun 16 '21 16:06 fejta-bot

Still a problem.

/remove-lifecycle rotten

kfox1111 avatar Jun 16 '21 16:06 kfox1111

/cc

pigletfly avatar Jul 13 '21 11:07 pigletfly

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue or PR as fresh with /remove-lifecycle stale
  • Mark this issue or PR as rotten with /lifecycle rotten
  • Close this issue or PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot avatar Oct 11 '21 12:10 k8s-triage-robot

/remove-lifecycle stale

kfox1111 avatar Oct 11 '21 16:10 kfox1111

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue or PR as fresh with /remove-lifecycle stale
  • Mark this issue or PR as rotten with /lifecycle rotten
  • Close this issue or PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot avatar Jan 09 '22 16:01 k8s-triage-robot

/remove-lifecycle stale

kfox1111 avatar Jan 10 '22 19:01 kfox1111

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue or PR as fresh with /remove-lifecycle stale
  • Mark this issue or PR as rotten with /lifecycle rotten
  • Close this issue or PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot avatar Apr 10 '22 19:04 k8s-triage-robot

/remove-lifecycle stale

kfox1111 avatar Apr 11 '22 16:04 kfox1111