Add Additional security groups to Cluster

[shashidharrao]

Why do you want this feature? My cluster end point is private API server endpoint and which is accessible only with in VPC and i want to add Additional security groups to the cluster during the Cluster setup so that i will be able administer the Cluster and EKS through my bastion host. I've enabled private end point access but stuck where i can't assign more security groups to the cluster to allow access from Jump-box.

What feature/behavior/change do you want? Add Additional security groups to Cluster Controlplane

[shashidharrao]

# An example of ClusterConfig object using an existing VPC:
---
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig

metadata:
  name: SP-NP-EKS
  region: us-east-1
  securityGroups:
      withShared: true
      withLocal: true
      attachIDs: ['sg-05c1f719382ab9279', 'sg-03305b8b8df813ede']
vpc:
  id: vpc-0e08dbb476ac7febc
  cidr: "10.0.0.0/16"
  subnets:
    # must provide 'private' and/or 'public' subnets by availibility zone as shown
    private:
      us-east-1c:
        id: "subnet-034b18d67e4348ac7"
        cidr: "10.0.2.0/24"

      us-east-1b:
        id: "subnet-00b7afab42694ebfd"
        cidr: "10.0.1.0/24"
    public:
      us-east-1a:
        id: "subnet-0905b9d19d87a0369"
        cidr: "10.0.0.0/24"

      us-east-1d:
        id: "subnet-0e2b092075fe64174"
        cidr: "10.0.3.0/24"
nodeGroups:
  - name: Frontend-ng
    instanceType: m5.xlarge
    desiredCapacity: 2
    maxSize: 10
    iam:
      attachPolicyARNs:
        - arn:aws:iam::aws:policy/AmazonDocDBReadOnlyAccess
       withAddonPolicies:
        autoScaler: true
    amiFamily: AmazonLinux2
    labels:
      nodegroup-type: Frontend
      instance-type: onDemand
    ssh:
      publicKeyName: EKSTESTING
    privateNetworking: true
    securityGroups:
      withShared: true
      withLocal: true
      attachIDs: ['sg-05c1f719382ab9279', 'sg-03305b8b8df813ede']
    preBootstrapCommands:
         - "#!/bin/bash"
         - "aws s3 cp s3://sp-cf-templates-us-east-1/NPRD/bootstrap/BootStrap.sh /root/BootStrap.sh"
         - "cat -v /root/BootStrap.sh | sed -e 's/\\^M//g' > /root/BootStrap1.sh"
         - "mv -f /root/BootStrap1.sh /root/BootStrap.sh"
         - "chmod +x /root/BootStrap.sh"
         - "sh /root/BootStrap.sh SP-NP-EKS-NG >> /root/log.txt 2>&1"

[aleksey-hariton]

:+1:

Having the same case, eksctl doc stating

You must ensure that your Amazon EKS control plane security group contains rules to allow ingress traffic on port 443 from your connected network.

But looks like there is no way to do that with eksctl or in cluster.yaml

[shashidharrao]

yeah you are correct i am adding the rules to security groups of cluster once it is provisioned. i am looking for a solution where i can add the rules during the cluster by assigning existing security groups to control plane. it is possible when i provision the cluster using cloud formation template but not with eksctl....

[aleksey-hariton]

@shashidharrao actually I found a way to do that, but you have to create SG before you creating cluster.

If you will specify that SG id in your cluster.yaml, option vpc.securityGroup, eksctl will not create its own SG to control access to ControlPlane, but use one you specified.

[aleksey-hariton]

In AWS Console you will see that SG as Additional security groups

[stateisbad]

+1 for the end result requested here. I would also like the possibility of appending to the auto-created ControlPlane SG with a list of CIDRs from the config yaml, i.e.:

• 10.0.0.0/8
• 192.168.1.0/24

and then those CIDRs are just appended/added to the created SG. No need to manage a separate group then, and everything is done in code.

[antoine-choimet-cbp]

Same problem here, I would like to automate the cluster creation with a gitlab runner running inside the same VPC. But when eksctl verify if control plane is available, the request timed out since the Cluster SG does not add the VPC CIDR in inbound access..

[shashidharrao]

vpc.securityGroup worked fine for me but i need to create one inbound rule to access the endpoint api from my VPC after cluster setup.

[lgg42]

Hi! We're also facing this problem, after many readings we thought EKS's private access would allow us to do this. In their docs they state:

Kubernetes API requests within your cluster's VPC (such as worker node to control plane communication) use the private VPC endpoint.

Then we found ourselves with this issue. We worked around this by adding with aws-cli the inboud rule in the Additional security group with our VPC CIDR. But we think this could really come better being part of eksctl

[michaelbeaumont]

This issue should address https://github.com/weaveworks/eksctl/issues/2627 as well

[CameronMcAuley]

For anyone who wants to automate this:

You can only add an Additional Security Group via passing the YAML config file, not in-line, and if you want to automate custom parameters you could use sed replacement on a template .yaml file.

See here for an example: https://github.com/weaveworks/eksctl/issues/2961

[aclevername]

Needs more research on all the different security group use cases

[bitva77]

You know how we have a publicAccessCIDRs list of IP's to allow access to the API? Why not a privateAccessCIDRs option that adds private IP's to the SG in question here.

It appears to be the security group named eks-cluster-sg-$ClusterName-$Random9DigitNumber that our private IP's need to be added to.

As it is, I'm having to manually add this after the fact using a bash script and some jq + grep + cut nonsense :)

[ar-bella-jeong]

I'm in the same state as @shashidharrao. Actually AWS strongly recommended that same control plain sg can't be used to various clusters.

In case of service account, it has not only attachRoleARN but also attachPolicyARNs. I really hope eksctl may provide the feature setting security group rules or specific inbound/outbound rules at the control plain sg in eksctl yaml.

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[github-actions[bot]]

This issue was closed because it has been stalled for 5 days with no activity.

[darmbrust]

I'm confused by the state of this feature request. Maybe its a documentation issue, but I can't figure out how to specify my "additional security groups" for a cluster.
The schema docs list: securityGroup: (aka the ControlPlaneSecurityGroup) for communication between control plane and nodes and sharedNodeSecurityGroup: for pre-defined shared node SG

Which really carries no distinction as to what does what.
The schema documentation only talks about sharedNodeSecurityGroup - and its relation to manageSharedNodeSecurityGroupRules - but says nothing about securityGroup.

What I'm looking for, is how I let eksctl create its managed security group, and let me add 2 additional pre-existing security groups.