Cannot create Ingress Gateways with the same name in different namespaces

[thomashashi]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request. Searching for pre-existing feature requests helps us consolidate datapoints for identical requirements into a single place, thank you!
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.

---

Overview of the Issue

When using the Helm chart version 0.42.0 (haven't been able to try it with the latest version) I am unable to create two ingress gateways with the same name but in different namespaces. I could do that manually if I were outside of Kubernetes, but cannot with the Helm chart — I believe we make some assumptions in how we name the Deployments and Pods created which don't take into account the namespace into which an IGW is provisioned.

Reproduction Steps

Create a Helm values file with the following stanza:

ingressGateways:
  enabled: true
  defaults:
    annotations: |
      "service.beta.kubernetes.io/aws-load-balancer-internal": "true"
      "service.beta.kubernetes.io/aws-load-balancer-scheme": "internal"
      "service.beta.kubernetes.io/aws-load-balancer-type": "nlb"
  gateways:
    - name: ingressgw
      consulNamespace: gateways
      service:
        type: ClusterIP
        ports:
          - port: 80
          - port: 443
    - name: ingressgw
      consulNamespace: k8s-122-gateways
      service:
        type: ClusterIP
        ports:
          - port: 80
          - port: 443

and attempt to deploy.

Logs

Output from helm upgrade consul hashicorp/consul -f consul-helm.yaml --namespace consul --version 0.42.0 (where consul-helm.yaml is the values file containing the stanza in the previous section):

Error: UPGRADE FAILED: template: consul/templates/ingress-gateways-deployment.yaml:18:3: executing "consul/templates/ingress-gateways-deployment.yaml" at <fail (cat "ingress gateways must have unique names but found duplicate name" $innerIngressVal.name)>: error calling fail: ingress gateways must have unique names but found duplicate name ingressgw

Expected behavior

Environment details

• Helm chart version 0.42.0
• Consul Enterprise 1.11.4

[jmurret]

This is the original commit from 2 years ago that introduced ingress gateways to consul k8s. An assumption and constraint about uniqueness is listed in the first paragraph:

By making use of `ingressGateways.defaults` and
`ingressGateways.gateways` setting, it's possible to enable multiple
ingress gateways. Names must be provided for each and they must be
unique.

We added a duplicate uniqueness check recently to the one that existed in the above commit: https://github.com/hashicorp/consul-k8s/commit/221d1096f59e16c2ff15e1dbe461d3b8b4c32ad4#diff-f4f5f85fa0d558e5e6f50fbc5353144059fc9d82209b78a72c90fd93c795750fR20

I am changing this from a bug to an enhancement but still looking into possibilities for a solution. cc: @david-yu

[thomashashi]

I would argue that this is still a bug: I can deploy Consul Enterprise outside of K8s and create as many gateways with the same name as I like, but in different namespaces. That, in fact, is one of the selling points of Consul Enterprise, that you can have services with the same name be treated as different services as long as they are registered in different namespaces. If I try to reproduce something which I can easily do, and which Consul Enterprise explicitly allows outside of K8s, and it fails, I'm not only seeing behavior unexpected from Consul Enterprise, but I'm also getting a substandard experience on Kubernetes.

If we are going to keep this requirement for the time being, can the Helm Chart documentation be updated to reflect that, even if you have Consul Enterprise, if you manage your Ingress Gateways via the Helm Chart they must have unique names even if they are in different namespaces? Commit notes are not documentation.

[david-yu]

This issue will still exist after https://github.com/hashicorp/consul-k8s/pull/2903 as we do not intend to change the behavior of Ingress Gateways as they are deprecated in favor of Consul API Gateway.