Flexible (anti-)affinity configuration in Helm chart

[pmichna]

Is your feature request related to a problem? Please describe. I'd like to have more power over how (anti-)affinity is configured for gateway-proxy and extauth. Currently it is hardcoded in the extauth deployment template and gateway-proxy deployment (with a simple switch for antiAffinity).

Describe the solution you'd like I'd like to be able to configure gateway-proxy, extauth and rate-limit podAntiAffinity and podAffinity without using kubeResourceOverride value.

Describe alternatives you've considered Currently it is possible with kubeResourceOverride but it's not convenient considering we havegloo.gatewayProxies.gatewayProxy.affinity as an interface but gloo.gatewayProxies.gatewayProxy.antiAffinity as bool.

Additional context I wanted to configure the following anti-affinity:

podAntiAffinity:
  preferredDuringSchedulingIgnoredDuringExecution:
    - weight: 100
      podAffinityTerm:
        labelSelector:
          matchLabels:
            tags.datadoghq.com/service: *extAuthServiceName
            tags.datadoghq.com/version: *glooVersion
        topologyKey: topology.kubernetes.io/zone
  requiredDuringSchedulingIgnoredDuringExecution:
    - labelSelector:
        matchLabels:
          tags.datadoghq.com/service: *extAuthServiceName
          tags.datadoghq.com/version: *glooVersion
      topologyKey: kubernetes.io/hostname

[chrisgaun]

Can run Envoy and ExtAuth as sidecar. This is supported today as a separate deployment which might be a potential workaround. Here are some helpful docs https://docs.solo.io/gloo-edge/latest/guides/security/auth/extauth/#switching-between-ext-auth-deployment-modes

Let's discuss more in channel.

[jmunozro]

Currently it is possible with kubeResourceOverride but it's not convenient considering we havegloo.gatewayProxies.gatewayProxy.affinity as an interface but gloo.gatewayProxies.gatewayProxy.antiAffinity as bool.

To give context about this: https://github.com/solo-io/gloo/blob/v1.8.17/install/helm/gloo/templates/7-gateway-proxy-deployment.yaml#L96

(additionally, maybe we should make it available to sidecar deployments too)

[soloio-bot]

Zendesk ticket #3044 has been linked to this issue.