wrong SG port for TargetGroupBindings in instance mode

[MartinEmrich]

Hello!

When configuring a targetGroupBinding for a Target Group in instance mode, it registers every K8s node with the service NodePort (31000 plus X).

But the security Groups generated are for the port of the service. Example:

apiVersion: v1
kind: Service
metadata:
  name: someservice
spec:
  selector:
    name: stateservice
  type: NodePort
  ports:
  - name: mqtt
    protocol: TCP
    port: 1883
    targetPort: 1883
---
apiVersion: elbv2.k8s.aws/v1beta1
kind: TargetGroupBinding
metadata:
  name: someservice
spec:
  serviceRef:
    name: someservice
    port: 1883
  targetGroupARN: arn:aws:elasticloadbalancing:eu-central-1:xxx
  # targetType: instance # setting this makes no difference
  networking:
    ingress:
    - from:
      - ipBlock:
          cidr: 10.41.0.0/16
      ports:
      - port: 1883
        protocol: TCP

Indeed it creates a security group rule allowing port 1883 from 10.41.0.0/16. But K8s assigns e.g. a nodeport of 31252 to the service, so the security group rule should use that port... or am I missing something?

The documentation (https://kubernetes-sigs.github.io/aws-load-balancer-controller/guide/targetgroupbinding/spec/#elbv2.k8s.aws/v1beta1.NetworkingPort) states " if port is unspecified, it defaults to all ports.". But if I set

     ports: []

No security group rule is created at all.

[MartinEmrich]

FYI: as a workaround, I specified the NodePort in the service and added it to the ports list. But I'd prefer the automatic port assignment by K8s, to avoid conflicts.

[kishorj]

If you are using NLB, then we don't support instance mode yet. Did you create the targetgroupbinding manually?

If you want to allow all ports, you can specify the following for the ports,

      ports:
      - protocol: TCP

We recommend more restricted security group rules.

[MartinEmrich]

Yes, I created the TargetGroupBinding manually... I did not know that Instance Mode ist not supported yet. But with my little workaround, it works fine... Are there any traps in front of me if I keep it that way? Will instance mode be supported in the future?

Until AWS EKS supports pod topology spread constraints, It's either instance mode or cross-zone load balancing to guarantee availability from every AZ....

Thanks, Martin

[fejta-bot]

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community. /lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity. Mark the issue as fresh with /remove-lifecycle rotten. Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community. /lifecycle rotten

[MartinEmrich]

/remove-lifecycle rotten

[fejta-bot]

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community. /lifecycle stale

[MartinEmrich]

/remove-lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[MartinEmrich]

/remove-lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[MartinEmrich]

/remove-lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[ranupratapsingh]

Hi all, I am also facing the same issue. I am trying to follow this guide. https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.4/guide/targetgroupbinding/targetgroupbinding/.

The targets are getting registered but the security group rules to accept the traffic from existing ALB is not getting added in the node's security group.

As a newbie to this, I figured that the work around is to have static NodePort in the service and open ingress rule on the basis of CIDR but this feels like hardcoding.

Is this still not supported in 2.4 ?

[ranupratapsingh]

Hi all, I figured this out from the spec doc section here.

We can specify networking to use source target group like this

  serviceRef:
    name: auth-prod-service # route traffic to the auth-prod-service
    port: 80
  targetGroupARN: arn:aws:elasticloadbalancing:ap-south-1:<your-tg-id>
  networking: 
    ingress:
      - from:
        - securityGroup: sg-<your-alb-sg-id>

[MartinEmrich]

Hmm then you still would have to know the ID of the autogenerated security group in advance, still looks to me as a workaround.

/remove-lifecycle rotten

[ranupratapsingh]

@MartinEmrich in this case, because ALB is and target groups both already created outside of kubernetes cluster, securityGroup ID is a static value much like targetGroupARN with reference to cluster.

But I agree this looks like workaround.

@kishorj instead of a security group ID, would an annotation/tag based selection of the ALB/SG a good suggestion to implement in the spec ?

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[MartinEmrich]

/remove-lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle rotten
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

[k8s-ci-robot]

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.