[EKS] [request]: Document EKS default launch template

[dimara]

trafficstars

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Tell us about your request

Extend EKS Launch template support guide to include the default launch template that EKS uses.

Which service(s) is this request for? EKS

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?

I am trying to create a managed node group with an extra EBS disk. Based on https://github.com/aws/containers-roadmap/issues/1199#issuecomment-748296234, I created a launch template with only BlockDeviceMappings in its launch template data. Here is the launch template data I used:

 {
    "BlockDeviceMappings": [
        {
            "DeviceName": "/dev/xvda",
            "Ebs": {
                "DeleteOnTermination": true,
                "VolumeSize": 200,
                "VolumeType": "gp2"
            }
        },
        {
            "DeviceName": "/dev/sdf",
            "Ebs": {
                "DeleteOnTermination": true,
                "VolumeSize": 500,
                "VolumeType": "gp2"
            }
        }
    ]
}

Then I created the node group using --launch-template:

aws eks create-nodegroup \
   --launch-template name=eks-nodegroup-ebs-extra-disk \
   --cluster-name ${CLUSTERNAME?} \
   --nodegroup-name ebs-extra-disk \
   --scaling-config minSize=0,maxSize=1,desiredSize=1 \
   --subnets ${SUBNET?} \
   --node-role ${EKS_WORKER_NODE_ROLE?} \
   --kubernetes-version 1.17 \
   --release-version=1.17.12-20210310 \
   --instance-types m5.xlarge

Then I inspected the generated launch template of the underlying ASG:

ASG=$(aws eks describe-nodegroup \
	--cluster-name ${CLUSTERNAME?} \
	--nodegroup-name ${NODEGROUP?} \
	--query nodegroup.resources.autoScalingGroups[] \
	--output text)
LT=$(aws autoscaling describe-auto-scaling-groups \
	--auto-scaling-group-name ${ASG?} \
	--query AutoScalingGroups[].MixedInstancesPolicy.LaunchTemplate.LaunchTemplateSpecification.LaunchTemplateName \
	--output text)
aws ec2 describe-launch-template-versions --launch-template-name ${LT}

Then I created another node group using the same options but without --launch-template. Comparing the launch templates of the underlying Auto Scaling Groups I found the following differences:

Default	Custom
"NetworkInterfaces": [{"DeviceIndex": 0, "Groups": ["sg-...."]}]	"SecurityGroupIds": ["sg-..."]
	"TagSpecifications": [{..."Tags":[{"key": "eks:cluster-name"..."key": "eks:nodegroup-name"}
"MetadataOptions": {"HttpPutResponseHopLimit": 2}

Copying from docs:

If you don't specify your own launch template to use when creating a managed node group, the Amazon EKS API creates a launch template with default values in your account.

Based on the above I have the following questions:

1. I would expect the only difference to be the BlockDeviceMappings. Why do are we seeing more?
2. What are the "default values" that EKS uses when creating launch templates?
3. In the default configuration, EKS sets HttpPutResponseHopLimit=2 which means that pods can assume the IAM role of the worker node. Shouldn't this be disabled by default?
4. Is it possible to have a custom launch template and have exactly the same outcome with the default one?

Are you currently working around this issue? In my custom launch template I include the MetadataOptions that EKS uses along with my BlockDeviceMapping.

Additional context I used an 1.17 EKS cluster and awscli 2.2.35.

Attachments None.

[psjamesh]

I would like to understand the default template specification too. Ideally I want to somehow "extend" the default template with my custom template specification. I am using CloudFormation.