Multi-Region Support

[ruiengana]

What would you like to be added: The ability to define different external-dns targets for each of the hostnames on the Ingress resource on a setup where each Ingress resource have multiple hostnames, e.g. a global hostname and a regional hostname.

Why is this needed: To simplify definition of multi-cluster / multi-region application deployment.

Assume the following multi-region infrastructure in AWS.

1 x AWS Global Accelerator - GA 2 x AWS Network Load Balancer - NLB 2 x AWS Elastic Kubernetes Service - EKS

GA instance.awsglobalaccelerator.com configured to target:

• NLB instance.elb.eu-west-1.amazonaws.com managed by ingress-nginx controller in eu-west-1 EKS
• NLB instance.elb.eu-central-1.amazonaws.com managed by ingress-nginx controller in eu-central-1 EKS

Both EKS clusters run independent external-dns and cert-manager workloads. All instances of external-dns are linked to same Route 53 zone example.com.

eu-west-1 EKS has following Ingress deployed

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt
  name: app
  namespace: default
spec:
  ingressClassName: nginx
  rules:
  - host: app.example.com
    http:
      paths:
      - path: /
        pathType: ImplementationSpecific
        backend:
          service:
            name: app
            port:
              number: 80
  - host: app.eu-west-1.example.com
    http:
      paths:
      - path: /
        pathType: ImplementationSpecific
        backend:
          service:
            name: app
            port:
              number: 80
  tls:
  - hosts:
    - app.example.com
    - app.eu-west-1.example.com
    secretName: app

eu-central-1 EKS has following Ingress deployed

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt
  name: app
  namespace: default
spec:
  ingressClassName: nginx
  rules:
  - host: app.example.com
    http:
      paths:
      - path: /
        pathType: ImplementationSpecific
        backend:
          service:
            name: app
            port:
              number: 80
  - host: app.eu-central-1.example.com
    http:
      paths:
      - path: /
        pathType: ImplementationSpecific
        backend:
          service:
            name: app
            port:
              number: 80
  tls:
  - hosts:
    - app.example.com
    - app.eu-central-1.example.com
    secretName: app

This will generate the following DNS entries in Route 53 (excluding the extra TXT entries for simplicity).

app.example.com CNAME instance.elb.eu-west-1.amazonaws.com <-- NOK - Flickering
app.example.com CNAME instance.elb.eu-central-1.amazonaws.com <-- NOK - Flickering
app.eu-west-1.example.com CNAME instance.elb.eu-west-1.amazonaws.com <-- OK
app.eu-central-1.example.com CNAME instance.elb.eu-central-1.amazonaws.com <-- OK

The desired outcome would be the following DNS records.

app.example.com CNAME instance.awsglobalaccelerator.com <-- OK
app.eu-west-1.example.com CNAME instance.elb.eu-west-1.amazonaws.com <-- OK
app.eu-central-1.example.com CNAME instance.elb.eu-central-1.amazonaws.com <-- OK

I understand we can split the Ingress resources so we have a single hostname per Ingress resource and annotate the Ingress managing the global hostname with external-dns.alpha.kubernetes.io/target to force external-dns point to the GA, but this isn't scalable for us where we have dozens of applications deployments in place. Since Ingress resource support multiple hostnames, we should be able to write external-dns ingress annotations applicable to a single hostname of the Ingress resource.

NOTE: Each instance cert-manager is managing a local Secret for the Certificate with global and regional hostnames as DNS names. The global hostname app.example.com and the regional hostname as app.eu-west-1.example.com or app.eu-central-1.example.com. We avoid any potential race condition with the different instances of cert-manager because DNS lists of each Certificate are unique per EKS cluster.

Hope this make sense.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle rotten
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

[k8s-ci-robot]

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.