Add external-dns with route53 support

[robinAwallace]

[!warning] This is a public repository, ensure not to disclose:

• [x] personal data beyond what is necessary for interacting with this pull request, nor
• [x] business confidential information, such as customer names.

What kind of PR is this?

Required: Mark one of the following that is applicable:

• [x] kind/feature
• [ ] kind/improvement
• [ ] kind/deprecation
• [ ] kind/documentation
• [ ] kind/clean-up
• [ ] kind/bug
• [ ] kind/other

Optional: Mark one or more of the following that are applicable:

[!important] Breaking changes should be marked kind/admin-change or kind/dev-change depending on type Critical security fixes should be marked with kind/security

• [ ] kind/admin-change
• [ ] kind/dev-change
• [ ] kind/security
• [ ] kind/adr

Release notes

Adds external-dns as a service to manage dns records present in aws route53.

What does this PR do / why do we need this PR?

This PR adds external-dns with route53 support. External-dns can mange dns records from kubernetes objects, such as service, ingress and DNSEndpoints. If enabled, it will only manage dns records it owns, referenced by txtOwnerId. So, it will not touch/take ownership of already created dns records.

• Fixes #280

Information to reviewers

I will put this PR as draft, as I still need to look at the following tasks. But it other then that, it works as intended.

• [x] Look at adding appropriate network polices
• [x] Look at adding migration script to allow external-dns to take over management of already existing dns records.

Checklist

• [x] Proper commit message prefix on all commits
• Change checks:
  • [ ] The change is transparent
  • [ ] The change is disruptive
  • [ ] The change requires no migration steps
  • [x] The change requires migration steps
  • [ ] The change upgrades CRDs
• Metrics checks:
  • [ ] The metrics are still exposed and present in Grafana after the change
  • [ ] The metrics names didn't change (Grafana dashboards and Prometheus alerts are not affected)
  • [ ] The metrics names did change (Grafana dashboards and Prometheus alerts were fixed)
• Logs checks:
  • [ ] The logs do not show any errors after the change
• Pod Security Policy checks:
  • [x] Any changed pod is covered by Pod Security Admission
  • [ ] Any changed pod is covered by Gatekeeper Pod Security Policies
  • [x] The change does not cause any pods to be blocked by Pod Security Admission or Policies
• Network Policy checks:
  • [x] Any changed pod is covered by Network Policies
  • [x] The change does not cause any dropped packets in the NetworkPolicy Dashboard
• Audit checks:
  • [ ] The change does not cause any unnecessary Kubernetes audit events
  • [ ] The change requires changes to Kubernetes audit policy
• Falco checks:
  • [ ] The change does not cause any alerts to be generated by Falco
• Bug checks:
  • [ ] The bug fix is covered by regression tests
• Config checks:
  • [ ] The schema was updated

[robinAwallace]

Should also the main README section for DNS be updated to reflect this change?

I can take a look at it.

Is it possible to create a new environment and let external-dns handle the creation of all the necessary DNS records during deployment? Im thinking that this might require some dependencies in the Helmfile stack.

Im not really sure what you mean here :thinking: I have not tested if external-dns is installed at the correct moment for new clusters. But it should work for new and existing clusters, if the migration script it used.

[anders-elastisys]

Is it possible to create a new environment and let external-dns handle the creation of all the necessary DNS records during deployment? Im thinking that this might require some dependencies in the Helmfile stack.

Im not really sure what you mean here 🤔 I have not tested if external-dns is installed at the correct moment for new clusters. But it should work for new and existing clusters, if the migration script it used.

I was referring to the need field in the helmfile stack, so that if you have externalDNS enabled it should be, as an example, a need for dex to ensure that the domain record is created before it is deployed. Would that make sense?

[robinAwallace]

Is it possible to create a new environment and let external-dns handle the creation of all the necessary DNS records during deployment? Im thinking that this might require some dependencies in the Helmfile stack.

Im not really sure what you mean here 🤔 I have not tested if external-dns is installed at the correct moment for new clusters. But it should work for new and existing clusters, if the migration script it used.

I was referring to the need field in the helmfile stack, so that if you have externalDNS enabled it should be, as an example, a need for dex to ensure that the domain record is created before it is deployed. Would that make sense?

Ah, yeah now i understand. Im not sure how the need hierarchy is planned. If we say dex needs external-dns, then it should also need ingress and cert-manager. But I can not see we have done those types of needs. @aarnq maybe have some input about this?

[robinAwallace]

Is it possible to create a new environment and let external-dns handle the creation of all the necessary DNS records during deployment? Im thinking that this might require some dependencies in the Helmfile stack.

Im not really sure what you mean here 🤔 I have not tested if external-dns is installed at the correct moment for new clusters. But it should work for new and existing clusters, if the migration script it used.

I was referring to the need field in the helmfile stack, so that if you have externalDNS enabled it should be, as an example, a need for dex to ensure that the domain record is created before it is deployed. Would that make sense?

Ah, yeah now i understand. Im not sure how the need hierarchy is planned. If we say dex needs external-dns, then it should also need ingress and cert-manager. But I can not see we have done those types of needs. @aarnq maybe have some input about this?

@anders-elastisys I had a talk with André about needs, and it is meant to be so lean as possible. So you can deploy a service without needing a lot of other components. So I will not add more needs in this PR. :slightly_smiling_face:

[aarnq]

Is it possible to create a new environment and let external-dns handle the creation of all the necessary DNS records during deployment? Im thinking that this might require some dependencies in the Helmfile stack.

Im not really sure what you mean here 🤔 I have not tested if external-dns is installed at the correct moment for new clusters. But it should work for new and existing clusters, if the migration script it used.

I was referring to the need field in the helmfile stack, so that if you have externalDNS enabled it should be, as an example, a need for dex to ensure that the domain record is created before it is deployed. Would that make sense?

Ah, yeah now i understand. Im not sure how the need hierarchy is planned. If we say dex needs external-dns, then it should also need ingress and cert-manager. But I can not see we have done those types of needs. @aarnq maybe have some input about this?

@anders-elastisys I had a talk with André about needs, and it is meant to be so lean as possible. So you can deploy a service without needing a lot of other components. So I will not add more needs in this PR. 🙂

Context in the last part of usage.

Also please update the local cluster support matrix if applicable :smile:

[robinAwallace]

@anders-elastisys @aarnq I have now done the changes asked for and added some more comments. PTAL :slightly_smiling_face: