[Feature Request] Per node-label configuration

[dbenque]

The Draino can be configured for a node selection thanks to --node-label parameter. In some kubernetes clusters, nodes can be labelled because they belong to different tenants or applications or because they run different workload types. For each of these dimensions we may require a different Draino configuration.

Would it be possible to run Draino with multiple configurations, each of them matching a --node-label selector?

I can see an alternative that would consist in running one Draino per --node-label but that would quickly be an operational overhead (+no resource sharing: one node informer per instance).

[jacobstr]

I'm going to walk this out a bit...

One way to achieve this would be to allow the use of a configuration file - otherwise parameters would get unwieldy. I think the approach here would be to take the existing "singleton" configuration from the command line args and to add the fields to a struct e.g.

type Config struct {
    NodeLabels []string
    Conditions []string
    // Arguably global configuration.
    ProtectedPodAnnotations []string
    EvictDaemonSetPods []string
    MaxGracePeriod time.Duration
    // ... 
}
type Configs []Config

Potentially, using the configuration file would be mutually exclusive with using the certain command line arguments - otherwise we need to reconcile the passed in args vs. the config file - I'd rather KIS.

---

Is that roughly what you had in mind? I think I'd accept an MR to implement this functionality but may be unlikely to get to it myself. I think the approach of having multiple drainos running is indeed how we envisioned this would work. It's certainly some additional bloat - but in my head I don't see really running that many instances. If folks have stories to share about say, approaching the need to run a half-dozen or more draino instances I'd be curious to hear.

[negz]

Would running multiple Drainos, each with a different configuration work for this use case?

[dbenque]

@negz well not really efficient for footprint when you have hundred of applications.

I was reading again the proposition of @jacobstr and was thinking that configFile could be discovered at runtime if we use configmaps. Each application that has different node-labels create a configMap named draino-config in its namespace. We give draino clusterRole on verb=Get Resource=Configmap and ResourceName=draino-config. Discovering this configuration and building the list of config would be dynamic.