external-dns icon indicating copy to clipboard operation
external-dns copied to clipboard

DNS not updated when load balancer changes address

Open pkit opened this issue 4 years ago • 24 comments
trafficstars

What happened: Contour logs

time="2021-09-20T17:37:52Z" level=info msg="received a new address for status.loadBalancer" context=loadBalancerStatusWriter loadbalancer-address=a09eed6b7d4769a328a1dc38396843-b3258fd74268025a.elb.us-east-1.amazonaws.com

external-dns logs:

time="2021-09-21T16:51:58Z" level=info msg="All records are already up to date"

Envoy service data:

AME    TYPE           CLUSTER-IP       EXTERNAL-IP                                                                     PORT(S)                      AGE   SELECTOR
envoy   LoadBalancer   10.100.146.240   a09eed6b7d4769a328a1dc38396843-b3258fd74268025a.elb.us-east-1.amazonaws.com   80:31483/TCP,443:32088/TCP   23h   app=envoy

What you expected to happen: Update of the new CNAME in Route53 Essentially that's the only thing I expect of external-dns nothing else.

How to reproduce it (as minimally and precisely as possible): Use external-dns with projectcontour on EKS

pkit avatar Sep 21 '21 16:09 pkit

It's much much worse than that. It always updates the address in Route53 to a wrong CNAME! That CNAME no longer exists in the cluster, yet external-dns will insist on setting it in Route53. Making the whole cluster unreachable. Only totally removing external-dns deployment can restore cluster functionality.

pkit avatar Sep 21 '21 17:09 pkit

The same happens when changing ingress class on a service (new ingress class has different lbip than the last). I was changing ingress class by deleting ingress for my service and creating new one with new ingress class. I expected ip to change. Provider: powerdns with txt records and traefik to traefik2 migration. On premise kubernetes 1.19, external dns from helm chart (5.4.10)

boniek83 avatar Oct 12 '21 14:10 boniek83

Reproduced again, and again, and again. It seems like external-dns just doesn't really handle load balancer address change at all. I.e. the one and only functionality it's needed for

pkit avatar Dec 02 '21 15:12 pkit

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue or PR as fresh with /remove-lifecycle stale
  • Mark this issue or PR as rotten with /lifecycle rotten
  • Close this issue or PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot avatar Mar 02 '22 15:03 k8s-triage-robot

/remove-lifecycle stale

kundan2707 avatar Mar 02 '22 17:03 kundan2707

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue or PR as fresh with /remove-lifecycle stale
  • Mark this issue or PR as rotten with /lifecycle rotten
  • Close this issue or PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot avatar May 31 '22 17:05 k8s-triage-robot

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue or PR as fresh with /remove-lifecycle rotten
  • Close this issue or PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

k8s-triage-robot avatar Jun 30 '22 18:06 k8s-triage-robot

/remove-lifecycle rotten

DerEnderKeks avatar Jun 30 '22 18:06 DerEnderKeks

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue or PR as fresh with /remove-lifecycle stale
  • Mark this issue or PR as rotten with /lifecycle rotten
  • Close this issue or PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot avatar Sep 28 '22 18:09 k8s-triage-robot

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue or PR as fresh with /remove-lifecycle rotten
  • Close this issue or PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

k8s-triage-robot avatar Oct 28 '22 19:10 k8s-triage-robot

/remove-lifecycle rotten

I hate this bot

DerEnderKeks avatar Oct 28 '22 21:10 DerEnderKeks

verified that external-dns:v0.12.2 does not update route 53 records when you change ingress from one ingress class to another one which has completely different load balancer hostname. I had to delete existing route 53 records to let external-dns create it again.

ismailyenigul avatar Nov 13 '22 18:11 ismailyenigul

you have to use --events flag

k0da avatar Nov 13 '22 20:11 k0da

@k0da added --events but did not help. Updated ingress class to new one and 30 min passed. external-dns did not update route 53 yet. external-dns pod arguments:

--log-level=info --log-format=text --interval=1m --events --source=service --source=ingress --policy=upsert-only --registry=txt --provider=aws

ismailyenigul avatar Nov 14 '22 07:11 ismailyenigul

@ismailyenigul did you happen to resolve this issue or figured out any work around ? i'm having the same issue and was wondering if there is any work around.

Arvindh-Mojo avatar Jan 05 '23 23:01 Arvindh-Mojo

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue as fresh with /remove-lifecycle stale
  • Close this issue with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot avatar Apr 05 '23 23:04 k8s-triage-robot

/remove-lifecycle stale

DerEnderKeks avatar Apr 06 '23 06:04 DerEnderKeks

@ismailyenigul did you happen to resolve this issue or figured out any work around ? i'm having the same issue and was wondering if there is any work around.

No solution by changing settings of external-dns but If I delete related records in route 53. External-dns created new records correctly.

ismailyenigul avatar Apr 06 '23 09:04 ismailyenigul

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue as fresh with /remove-lifecycle stale
  • Close this issue with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot avatar Jul 05 '23 09:07 k8s-triage-robot

/remove-lifecycle stale

dparker2 avatar Jul 12 '23 05:07 dparker2

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue as fresh with /remove-lifecycle stale
  • Close this issue with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot avatar Jan 24 '24 07:01 k8s-triage-robot

/remove-lifecycle stale

DerEnderKeks avatar Jan 24 '24 11:01 DerEnderKeks

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue as fresh with /remove-lifecycle stale
  • Close this issue with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot avatar Apr 23 '24 11:04 k8s-triage-robot

/remove-lifecycle stale

DerEnderKeks avatar Apr 23 '24 16:04 DerEnderKeks

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue as fresh with /remove-lifecycle stale
  • Close this issue with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot avatar Jul 22 '24 16:07 k8s-triage-robot

/remove-lifecycle stale

DerEnderKeks avatar Jul 22 '24 16:07 DerEnderKeks

We have the same issue with the CloudFlare provider. Is there any plans to give this a look? In other words making external-dns support updating DNS records on providers in the case that the source get its IP value changed/updated?

Thank you very much.

larssb avatar Sep 05 '24 15:09 larssb