aws-load-balancer-controller icon indicating copy to clipboard operation
aws-load-balancer-controller copied to clipboard

Published 20 hours ago • verified publisher icon kubernetes-sigs

swapping subnet position works

Open tooptoop4 opened this issue 2 years ago • 5 comments

my k8s ingress object definition had:

  annotations:
    alb.ingress.kubernetes.io/certificate-arn: redact
    alb.ingress.kubernetes.io/group.name: infra-apps
    alb.ingress.kubernetes.io/healthcheck-path: /swaggerui/
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":443}]'
    alb.ingress.kubernetes.io/load-balancer-attributes: idle_timeout.timeout_seconds=4000
    alb.ingress.kubernetes.io/scheme: internal
    alb.ingress.kubernetes.io/security-groups: redacty,redactz
    alb.ingress.kubernetes.io/ssl-policy: ELBSecurityPolicy-TLS-1-2-2017-01
    alb.ingress.kubernetes.io/subnets: subnet-redact1,subnet-redact2,subnet-redact3
    alb.ingress.kubernetes.io/tags: redact
    alb.ingress.kubernetes.io/target-type: instance
    external-dns.alpha.kubernetes.io/alias: "true"
    external-dns.alpha.kubernetes.io/hostname: redact

load-balancer controller pod logs showing:

{"level":"info","ts":1656988783.3797832,"logger":"controllers.ingress","msg":"creating loadBalancer","stackID":"infra-apps","resourceID":"LoadBalancer"}
{"level":"error","ts":1656988783.5488162,"logger":"controller-runtime.manager.controller.ingress","msg":"Reconciler error","name":"infra-apps","namespace":"","error":"InvalidSubnet: Not enough IP space available in subnet-redact1. ELB requires at least 8 free IP addresses in each subnet.\n\tstatus code: 400, request id: 46redact"}
{"level":"debug","ts":1656988783.5488386,"logger":"controller-runtime.manager.events","msg":"Warning","object":{"kind":"Ingress","namespace":"argocd","name":"argocd-ingress","uid":"c6redact","apiVersion":"networking.k8s.io/v1","resourceVersion":"54739886"},"reason":"FailedDeployModel","message":"Failed deploy model due to InvalidSubnet: Not enough IP space available in subnet-redact1. ELB requires at least 8 free IP addresses in each subnet.\n\tstatus code: 400, request id: 46redact"}

then i changed the order of values in subnets tag like so:

  annotations:
    alb.ingress.kubernetes.io/certificate-arn: redact
    alb.ingress.kubernetes.io/group.name: infra-apps
    alb.ingress.kubernetes.io/healthcheck-path: /swaggerui/
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":443}]'
    alb.ingress.kubernetes.io/load-balancer-attributes: idle_timeout.timeout_seconds=4000
    alb.ingress.kubernetes.io/scheme: internal
    alb.ingress.kubernetes.io/security-groups: redacty,redactz
    alb.ingress.kubernetes.io/ssl-policy: ELBSecurityPolicy-TLS-1-2-2017-01
    alb.ingress.kubernetes.io/subnets: subnet-redact2,subnet-redact1,subnet-redact3
    alb.ingress.kubernetes.io/tags: redact
    alb.ingress.kubernetes.io/target-type: instance
    external-dns.alpha.kubernetes.io/alias: "true"
    external-dns.alpha.kubernetes.io/hostname: redact

and the pod logs started showing successful creation of my loadbalance and listeners. i also verified that subnet-redact1 was still full.

so i think rather than me manually changing different positions, can the code try each subnet rather than exiting out if the first is lacking IP addresses?

tooptoop4 avatar Jul 05 '22 04:07 tooptoop4

@tooptoop4, in case you reorder the subnets, does the created ALB contain subnets that are full?

The subnet auto-discovery feature [live-doc], excludes subnets without enough IP addresses. In case of manually specified subnets, controller doesn't perform further validation, the error is from the AWS ELBv2 APIs. If create load balancer API validates only the first subnet for free IP addresses, and ignores the remaining ones, then we need to investigate it further.

kishorj avatar Jul 12 '22 16:07 kishorj

when I re-order, the created ALB contains the 3 subnets (2 with free space, 1 full) and it works well (listener/target group rules work and I can access my UI endpoints)

tooptoop4 avatar Jul 12 '22 18:07 tooptoop4

@tooptoop4 This seems to be bug in ALB side.
Thanks for reporting this to us, we'll sync with ALB team internally on this one.

M00nF1sh avatar Jul 14 '22 17:07 M00nF1sh

I am interested in solving this bug. It will be request you to allot me this issue to solve it. @M00nF1sh @tooptoop4 @kishorj @EppO @mortent

ArpanShah2k avatar Sep 11 '22 06:09 ArpanShah2k

@ArpanShah2k, thanks for your interest, I will assign it to you.

/assign @ArpanShah2k

kishorj avatar Sep 12 '22 16:09 kishorj

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue or PR as fresh with /remove-lifecycle stale
  • Mark this issue or PR as rotten with /lifecycle rotten
  • Close this issue or PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot avatar Dec 11 '22 17:12 k8s-triage-robot

/remove-lifecycle stale

tooptoop4 avatar Dec 11 '22 20:12 tooptoop4

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue as fresh with /remove-lifecycle stale
  • Close this issue with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot avatar Mar 11 '23 20:03 k8s-triage-robot

/remove-lifecycle stale

tooptoop4 avatar Mar 11 '23 20:03 tooptoop4

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue as fresh with /remove-lifecycle stale
  • Close this issue with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot avatar Jun 09 '23 21:06 k8s-triage-robot

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue as fresh with /remove-lifecycle rotten
  • Close this issue with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

k8s-triage-robot avatar Jul 09 '23 22:07 k8s-triage-robot

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Reopen this issue with /reopen
  • Mark this issue as fresh with /remove-lifecycle rotten
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

k8s-triage-robot avatar Jan 19 '24 14:01 k8s-triage-robot

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Reopen this issue with /reopen
  • Mark this issue as fresh with /remove-lifecycle rotten
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

k8s-ci-robot avatar Jan 19 '24 14:01 k8s-ci-robot