External-DNS selects only one network loadbalancer on Route53

[abbabe]

trafficstars

I use External-DNS in AWS EKS. Internal and internet-facing (external) network loadbalancer is installed on AWS with Istio Service. It has the necessary roles and policies to create dns records in private hostzone on Route 53 with External-DNS service account. However, the problem is that External-DNS selects external network load balancer when creating the host address registered in my virtual service.yaml file on Route 53. I actually want it to select the internal network loadbalancer. Although I gave values such as loadbalancer annotations and label, I could not solve the problem. How can I direct the loadbalancer selection process of External DNS on route 53 to the NLB I want. Is it possible to do this?

[ivankatliarchuk]

This description doesn't provide enough context for to understand the problem. It appears to rely on knowledge of a specific environment that isn't shared with exceptions that random folks from internet aware of it.

To help reproduce the issue and potentially find a solution, could you provide the full setup details like kubernetes manifests, terraform configs, how to create load balancers, arguments (not helm) set on external dns, version of exernal dns and etc?

[abbabe]

I attached the setup files. 1- EXTERNAL DNS values.yaml 2- EXTERNAL NLB 3- INTERNAL NLB 4- TERRAFORM IRSA

When I assign a dns as test.int.dagi.com in the dns definitions I made on istio, the external loadbalancer is automatically selected on route53, not the internal network load balancer.

A correct selection between the two loadbalancers was not possible at this stage.

Thanks in advance

EXTERNAL DNS values.yaml

serviceAccount:
  create: true
  annotations: 
    eks.amazonaws.com/role-arn: arn:aws:iam::xxxxxxxxxxx:role/external-dns-sa-role
  name: external-dns-sa

rbac:
  # -- If `true`, create a `ClusterRole` & `ClusterRoleBinding` with access to the Kubernetes API.
  create: true

# -- Interval for DNS updates.
interval: 1m

# -- _Kubernetes_ resources to monitor for DNS entries.
sources:
  - service
  - ingress
  - istio-gateway
  - istio-virtualservice 

# -- How DNS records are synchronized between sources and providers; available values are `sync` & `upsert-only`.
policy: upsert-only

# -- Specify the registry for storing ownership and labels.
# Valid values are `txt`, `aws-sd`, `dynamodb` & `noop`.
registry: noop
# -- (string) Specify an identifier for this instance of _ExternalDNS_ wWhen using a registry other than `noop`.
txtOwnerId: external-dns

## - Limit possible target zones by domain suffixes.
domainFilters:
  - int.dagi.com

provider:
  name: aws

# -- Extra arguments to provide to _ExternalDNS_.
extraArgs: 
  - --aws-zone-type=private   #  route53 zone type private
  - --label-filter=external-dns=internal
  - --annotation-filter=external-dns.alpha.kubernetes.io/visibility=internal

`

ISTIO Service EXTERNAL NETWORK LOAD BALANCER

--- 
apiVersion: v1
kind: Service
metadata:
  name: istio-ingressgateway-external-nlb
  namespace: istio-system
  labels:
    app: istio-ingressgateway
    app.kubernetes.io/name: istio-ingressgateway
    external_dns: external
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-name: k8s-istio-ingress-ext-nlb
    service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
    service.beta.kubernetes.io/aws-load-balancer-healthcheck-path: /healthz/ready
    service.beta.kubernetes.io/aws-load-balancer-healthcheck-port: status-port
    service.beta.kubernetes.io/aws-load-balancer-healthcheck-protocol: HTTP
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: ssl
    service.beta.kubernetes.io/aws-load-balancer-ssl-cert: "arn:aws:acm:us-west-2:XXXXXXXXXXXX:certificate/edrftgyhuj-54b8-4436-8f56-67ft8mk9sgn9"
    service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443"
    service.beta.kubernetes.io/aws-load-balancer-ssl-negotiation-policy: ELBSecurityPolicy-TLS13-1-2-2021-06
    service.beta.kubernetes.io/aws-load-balancer-attributes: "access_logs.s3.enabled=true,access_logs.s3.bucket=dev-lb-logs-us-west-2-XXXXXXXXXXXX,access_logs.s3.prefix=k8s-istio-ingress-ext-nlb,deletion_protection.enabled=true"
    service.beta.kubernetes.io/aws-load-balancer-additional-resource-tags: "Environment=dev,Provisioner=Kubernetes, external_dns=external"
spec:
  ports:
    - name: status-port
      protocol: TCP
      port: 15021
      targetPort: 15021
    - name: https
      protocol: TCP
      port: 443
      targetPort: 443

  selector:
    app: istio-ingressgateway
    istio: ingressgateway
  type: LoadBalancer

ISTIO Service INTERNAL NETWORK LOAD BALANCER

apiVersion: v1
kind: Service
metadata:
  name: istio-ingressgateway-nlb
  namespace: istio-system
  labels:
    app: istio-ingressgateway
    app.kubernetes.io/name: istio-ingressgateway
    external-dns: internal
  annotations:
    external-dns.alpha.kubernetes.io/visibility: "internal" # to select internal nlb on route53 for external dns
    service.beta.kubernetes.io/aws-load-balancer-name: k8s-istio-ingress-int-nlb
    service.beta.kubernetes.io/aws-load-balancer-scheme: internal
    service.beta.kubernetes.io/aws-load-balancer-healthcheck-path: /healthz/ready
    service.beta.kubernetes.io/aws-load-balancer-healthcheck-port: status-port
    service.beta.kubernetes.io/aws-load-balancer-healthcheck-protocol: HTTP
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: ssl
    service.beta.kubernetes.io/aws-load-balancer-ssl-cert: "arn:aws:acm:us-west-2:XXXXXXXXXXXX:certificate/edrftgyhuj-54b8-4436-8f56-67ft8mk9sgn9""
    service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443"
    service.beta.kubernetes.io/aws-load-balancer-ssl-negotiation-policy: ELBSecurityPolicy-TLS13-1-2-2021-06
    service.beta.kubernetes.io/load-balancer-source-ranges: "10.10.0.0/16"
    service.beta.kubernetes.io/aws-load-balancer-attributes: "access_logs.s3.enabled=true,access_logs.s3.bucket=dev-lb-logs-us-west-2-XXXXXXXXXXXX,access_logs.s3.prefix=k8s-istio-ingress-int-nlb,deletion_protection.enabled=true"
    service.beta.kubernetes.io/aws-load-balancer-additional-resource-tags: "Environment=dev,Provisioner=Kubernetes"
spec:
  ports:
    - name: status-port
      protocol: TCP
      port: 15021
      targetPort: 15021
    - name: http2
      protocol: TCP
      port: 80
      targetPort: 80
    - name: https
      protocol: TCP
      port: 443
      targetPort: 443
  selector:
    app: istio-ingressgateway
    istio: ingressgateway
  type: LoadBalancer

Terraform IRSA (IAM Roles for Service Accounts)

---

##################### external-dns ##################

data "aws_iam_policy_document" "external_dns_iam_policy" {
  statement {
    sid       = ""
    effect    = "Allow"
    resources = ["arn:aws:route53:::hostedzone/*"]
    actions   = ["route53:ChangeResourceRecordSets"]
  }

  statement {
    sid       = ""
    effect    = "Allow"
    resources = ["*"]

    actions = [
      "route53:ListHostedZones",
      "route53:ListResourceRecordSets",
      "route53:ListTagsForResource",
    ]
  }
}

##################### external-dns ##################

module "external_dns_iam_policy" {
  source  = "terraform-aws-modules/iam/aws//modules/iam-policy"
  version = "5.39.0"
  providers = {
    aws = aws.target
  }

  name        = "external-dns-sa-policy"
  path        = "/"
  description = "Allow the External DNS in EKS to access the Route53"
  policy      = data.aws_iam_policy_document.external_dns_iam_policy.json
  tags        = var.tags
}

module "external_dns_iam_eks_role" {
  source  = "terraform-aws-modules/iam/aws//modules/iam-role-for-service-accounts-eks"
  version = "5.39.0"
  providers = {
    aws = aws.target
  }

  role_name = "external-dns-sa-role"
  role_policy_arns = {
    policy = module.external_dns_iam_policy.arn
  }

  oidc_providers = {
    one = {
      provider_arn               = var.eks_oidc_provider_arn
      namespace_service_accounts = var.external_dns_namespace_service_account_pairs
    }
  }

  tags = var.tags
}

[ivankatliarchuk]

Any chacne to format code, you could use code block <> or "```" /assign

[ivankatliarchuk]

And not clear which version you are on

[abbabe]

And not clear which version you are on

I edited my code blog and the version I'm using is:

sources: 
    - repoURL: 'https://kubernetes-sigs.github.io/external-dns/'
      targetRevision: **1.14.4**
      chart: external-dns
      helm:

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[ivankatliarchuk]

/assign

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle rotten
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten