kubernetes plugin: Integrate with configmaps?

[rata]

Hi,

We are using configmaps in our pods. The basic way to deploy with configmaps is to create a new one on each deploy, reference it in the deployment yaml and deploy. This way, changes to configmaps are deployed too. And in kube 1.3 it is expected that support to garbage collect unused configmaps will be added.

Is there any way you had in mind to integrate configmaps in samson?

Or at least a way to run some custom commands on the deploy, so we can manage the configmaps manually till support for confimaps is properly integrated (we probably can help with the implementation)

[jonmoter]

Hi rata. I'm interested in doing something with ConfigMaps, but haven't looked deeply enough into them to have a plan.

How were you thinking of using them? Would you want to have a UI to enter config information that's stored in Samson, and then create ConfigMaps from that? Or would have you have one or more YAML files in the repo containing that information? Or some other way?

In general, I'm curious about your plans for using the Samson/Kubernetes integration. As you've probably noticed (and the Readme) says, it's still under active development and isn't complete yet. So I'd be interested to collaborate with you on ideas and possibly implementation. We have some specific needs here at Zendesk, but I'd like the integration to be as generally useful as possible.

You can shoot me an email if you'd like, [email protected]

[rata]

On Tue, Apr 19, 2016 at 02:39:37PM -0700, Jon Moter wrote:

Hi rata. I'm interested in doing something with ConfigMaps, but haven't looked deeply enough into them to have a plan.

How were you thinking of using them? Would you want to have a UI to enter config information that's stored in Samson, and then create ConfigMaps from that? Or would have you have one or more YAML files in the repo containing that information? Or some other way?

I'd like to use confimaps for nginx, for example, at a start. We have an nginx image and all pods that want/need an nginx, use that image and a custom config file via configmaps. The same for a log collector, for example.

And probably in some months, move our app specific configuration to configmaps+secrets.

I'm not sure how this can be handled nicely from a UI. One way for cases like "create a configmap from this/these file/files" can be to have in the UI a mapping from file/files to configmaps to create and reference on deployment yamls on deploy. I think this use case is useful enough.

A new configmap should be created when we want to change something, referenced in the pod yaml and deployed normally (as any deploy). This is explained here: https://github.com/kubernetes/kubernetes/issues/22368. But basically, old pods use the old configmap and new ones the new one, so no update should be done. And using a deploy avoids a syntax error being too serious, as pods fail and deploy doesn't continue. And a rollback can be done because the old configmap still exists and wasn't modified.

In k8s 1.3 configmaps will be "garbage collected", so it shouldn't be an issue to have many flying around. And meanwhile probably a pod can do that work, or something like that.

However I'm curious to see how other UIs will handle this. Kubernetes dashboard has no way to do it yet, it's being redesigned (and right now you can't even do a deployment, so it's really limited now). Do you know of other UIs? Have you tried any?

In general, I'm curious about your plans for using the Samson/Kubernetes integration. As you've probably noticed (and the Readme) says, it's still under active development and isn't complete yet. So I'd be interested to collaborate with you on ideas and possibly implementation. We have some specific needs here at Zendesk, but I'd like the integration to be as generally useful as possible.

You can shoot me an email if you'd like, [email protected]

Thanks a lot! I'll do, but tomorrow because it's 10pm here :-D

[jonmoter]

At Zendesk, we have multiple environments and datacenters, so a lot of times our config values change depending on the datacenter. For example, we might deploy the same app to multiple AWS regions, and one of the config variables might be the name of the AWS region, or the location of an S3 bucket. In Samson, we model the different datacenters as a different "Deploy Group". And each datacenter would have a separate Kubernetes cluster running in it.

That means that if we were to create ConfigMaps, the values frequently might be different on a per-datacenter basis. So having a single YAML file in the repo wouldn't work for us. Maybe a separate config file for each deploy group? Or some mechanism to specify overrides?

One option would be to add a ConfigMap entry to the YAML file that specifies the Deployment and the Service for the project. Then it would be a little extra logic to generate a ConfigMap through the API for each Kubernetes::ReleaseDoc that got created, and associate the two. But I haven't thought through this deeply.

[rata]

Thanks a lot!

I think if you need different configurations on different datacentes, then you will need different configmaps on different datacenter (if you want to use configmaps for those configs).

And about specifying the configmap in the YAML that specifies the deployment, I think that can work (although using the "--from-file" option sometimes is easier, specially when you use it with directories) as long as a new configmap object gets created each time and referenced in the pod. And, of course, the option to have more than one configmap in the YAML and referenced in the pod (for example, different configs for different containers in the pod).

The con of that option is that the YAML file can become quite large, not so a happy place to edit, and I don't know if using a big YAML file can create any issue with some size limits in PUT api-server methods or something like that.