containers-roadmap
containers-roadmap copied to clipboard
aws
[EKS] [request]: EKS nodegroup list AMI releases API
Community Note
- Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
- Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
- If you are interested in working on this issue or have submitted a pull request, please leave a comment
Tell us about your request
Expose an API that will support listing the AMI release versions for a given Kubernetes version and AMI type (AL2, Bottlerocket, Windows variants, etc.), similar to the describe addon versions API (aws eks describe-addon-versions --kubernetes-version 1.22 --addon-name coredns)
Psuedo AWSCLI
aws eks describe-ami-releases --kubernetes-version 1.22 --ami-type AL2_x86_64
Psuedo Terrfaorm
data "aws_eks_ami_release_version" "default" {
ami_type = "AL2_x86_64"
kubernetes_version = "1.22"
}
data "aws_eks_ami_release_version" "latest" {
ami_type = "AL2_x86_64"
kubernetes_version = "1.22"
most_recent = true
}
Which service(s) is this request for? EKS
Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?
When users deploy a cluster today with EKS managed node groups, when not using a custom AMI the AMI selected is based on the AMI type selected and the corresponding latest AMI release version is selected. Since AWS continuously update/patches AMIs, there are new AMI releases that come available but do not affect the nodegroup unless a user tracks down the latest AMI release version and specifies that in their nodegroup configuration. Users would like the ability to specify something along the lines of use the latest AMI release version without needing to know what that specific version is; they just want to ensure their fleet of instances are up to date and patched of any CVEs. To work around this, some customers are using the SSM parameter where AWS publishes the AMI IDs for EKS AMIs and passing this AMI ID into a custom launch template. This is not ideal because this is now treated as a "custom AMI" even though it is not custom - simply providing an AMI ID to the nodegroup API though will assume it is custom and therefore customers must provide the user data to bootstrap and join nodes to the cluster.
If an API similar to the describe-addon-versions existed, this API can be used to retrieve the latest AMI release and pass that to the nodegroup which will roll out the latest AMI release without any additional steps for user data bootstrapping, etc.
Are you currently working around this issue?
- Some customers are pulling the AMI ID from the AWS SSM parameter but this requires additional work to ensure bootstrap user data is re-added
Additional context
- https://github.com/terraform-aws-modules/terraform-aws-eks/issues/2434
- https://github.com/terraform-aws-modules/terraform-aws-eks/issues/2244
- https://github.com/terraform-aws-modules/terraform-aws-eks/issues/907
- https://github.com/hashicorp/terraform-provider-aws/issues/12675
Attachments If you think you might have additional information that you'd like to include via an attachment, please do - we'll take a look. (Remember to remove any personally-identifiable information.)
You can already do this with the CLI for AL2 & Bottlerocket as documented in the EKS guide. You can also easily do this in Terraform if you know the AMI naming structure with the aws_ami data source.
data "aws_ami" "al2_amd64_1_24" {
owners = ["amazon"]
filter {
name = "name"
values = ["amazon-eks-node-1.24-v*"]
}
most_recent = true
}
data "aws_ami" "al2_arm64_1_24" {
owners = ["amazon"]
filter {
name = "name"
values = ["amazon-eks-arm64-node-1.24-v*"]
}
most_recent = true
}
data "aws_ami" "bottlerocket_amd64_1_24" {
owners = ["amazon"]
filter {
name = "name"
values = ["bottlerocket-aws-k8s-1.24-x86_64-v*"]
}
most_recent = true
}
data "aws_ami" "bottlerocket_arm64_1_24" {
owners = ["amazon"]
filter {
name = "name"
values = ["bottlerocket-aws-k8s-1.24-aarch64-v*"]
}
most_recent = true
}
Thank you but I don't believe so - I can definitely get the AMI ID, but what I am after is the release version that is used by EKS managed nodegroups so that I don't need to treat it as a custom AMI
Here is the output of the data source for example
output = {
"architecture" = "x86_64"
"arn" = "arn:aws:ec2:us-east-1::image/ami-00c876151c8d1cb24"
"block_device_mappings" = toset([
{
"device_name" = "/dev/xvda"
"ebs" = tomap({
"delete_on_termination" = "true"
"encrypted" = "false"
"iops" = "0"
"snapshot_id" = "snap-0c3eeea262ace1864"
"throughput" = "0"
"volume_size" = "20"
"volume_type" = "gp2"
})
"no_device" = ""
"virtual_name" = ""
},
])
"boot_mode" = ""
"creation_date" = "2023-02-01T20:43:29.000Z"
"deprecation_time" = "2025-02-01T20:43:29.000Z"
"description" = "EKS Kubernetes Worker AMI with AmazonLinux2 image, (k8s: 1.22.17, docker: 20.10.17-1.amzn2.0.1, containerd: 1.6.6-1.amzn2.0.2)"
"ena_support" = true
"executable_users" = tolist(null) /* of string */
"filter" = toset([
{
"name" = "name"
"values" = tolist([
"amazon-eks-node-1.22-v*",
])
},
])
"hypervisor" = "xen"
"id" = "ami-00c876151c8d1cb24"
"image_id" = "ami-00c876151c8d1cb24"
"image_location" = "amazon/amazon-eks-node-1.22-v20230127"
"image_owner_alias" = "amazon"
"image_type" = "machine"
"imds_support" = ""
"include_deprecated" = false
"kernel_id" = ""
"most_recent" = true
"name" = "amazon-eks-node-1.22-v20230127"
"name_regex" = tostring(null)
"owner_id" = "602401143452"
"owners" = tolist([
"amazon",
])
"platform" = ""
"platform_details" = "Linux/UNIX"
"product_codes" = toset([])
"public" = true
"ramdisk_id" = ""
"root_device_name" = "/dev/xvda"
"root_device_type" = "ebs"
"root_snapshot_id" = "snap-0c3eeea262ace1864"
"sriov_net_support" = "simple"
"state" = "available"
"state_reason" = tomap({
"code" = "UNSET"
"message" = "UNSET"
})
"tags" = tomap({})
"timeouts" = null /* object */
"tpm_support" = ""
"usage_operation" = "RunInstances"
"virtualization_type" = "hvm"
}
Where I can see things like
"name" = "amazon-eks-node-1.22-v20230127"`
"image_location" = "amazon/amazon-eks-node-1.22-v20230127"
But the only place I see the patch version is in the description field. Match that up with a describe call on a nodegroup:
{
"nodegroup": {
"nodegroupName": "default-2023020613565708130000002f",
"nodegroupArn": "arn:aws:eks:us-east-1:xxxx:nodegroup/test-mixed/default-2023020613565708130000002f/5cc31381-fc59-1f39-3a01-28c857551198",
"clusterName": "test-mixed",
"version": "1.22",
"releaseVersion": "1.22.17-20230127",
"createdAt": "2023-02-06T08:56:58.750000-05:00",
"modifiedAt": "2023-02-06T11:37:33.730000-05:00",
"status": "ACTIVE",
"capacityType": "ON_DEMAND",
"scalingConfig": {
"minSize": 1,
"maxSize": 3,
"desiredSize": 1
},
"instanceTypes": [
"t3.medium"
],
"subnets": [
"subnet-0dd8a0e25eb75c830",
"subnet-0ca4af04a3cdc7033",
"subnet-01282234cc283cf60"
],
"amiType": "AL2_x86_64",
"nodeRole": "arn:aws:iam::xxxx:role/default-eks-node-group-20230206134356980700000001",
"labels": {},
"resources": {
"autoScalingGroups": [
{
"name": "eks-default-2023020613565708130000002f-5cc31381-fc59-1f39-3a01-28c857551198"
}
]
},
"diskSize": 20,
"health": {
"issues": []
},
"updateConfig": {
"maxUnavailablePercentage": 33
}
}
}
You can see the format that release version takes
"releaseVersion": "1.22.17-20230127",
And that is what I am after so I can provide it to the nodegroup to keep it up to date. I could potentially get there from some string manipulation but I need the patch version since its in the release version, and I don't know where to get that today reliably
There is a S3 API which up until I tested it again today was reliable (for some reason v20230127 has overwritten v20230111. The following will print all of the versions from a build date, and it's simple to switch up the matching to behave the same as the other APIs I added above.
eks_build_date="2023-01-11"
aws s3 ls s3://amazon-eks/ --recursive | grep -oE "[0-9]+\\.[0-9]+\\.[0-9]+/${eks_build_date}" | uniq | grep -oE '[0-9]+\.[0-9]+\.[0-9]+'
Slight change in spec, the build date is for the binary components so it doesn't change every release but is documented on the AMI release page. This same data is also available from SSM for AL2 as release_version.
I can't remember how we used to do this for Bottlerocket MNGs (we use an AMI & LT always now) or if it expects a different version format. I suspect this data might be harder to get at.
I am using this as a workaround.
locals {
cluster_version = "1.22"
data "aws_ssm_parameter" "eks_default_bottlerocket_image_version" {
name = "/aws/service/bottlerocket/aws-k8s-${local.cluster_version}/x86_64/latest/image_version"
}
then consume it like this:
eks_managed_node_groups = {
general = {
ami_type = "BOTTLEROCKET_x86_64"
ami_release_version = nonsensitive(data.aws_ssm_parameter.eks_default_bottlerocket_image_version.value) # e.g. v1.12.0-6ef1139f
here is the ssm content for context:
aws ssm get-parameters-by-path --path /aws/service/bottlerocket/aws-k8s-1.22/x86_64/latest/ --region us-east-1 --recursive
{
"Parameters": [
{
"Name": "/aws/service/bottlerocket/aws-k8s-1.22/x86_64/latest/image_id",
"Type": "String",
"Value": "ami-0876c047d0b186fcb",
"Version": 15,
"LastModifiedDate": "2023-01-26T20:34:32.459000-08:00",
"ARN": "arn:aws:ssm:us-east-1::parameter/aws/service/bottlerocket/aws-k8s-1.22/x86_64/latest/image_id",
"DataType": "text"
},
{
"Name": "/aws/service/bottlerocket/aws-k8s-1.22/x86_64/latest/image_version",
"Type": "String",
"Value": "1.12.0-6ef1139f",
"Version": 15,
"LastModifiedDate": "2023-01-26T20:34:33.303000-08:00",
"ARN": "arn:aws:ssm:us-east-1::parameter/aws/service/bottlerocket/aws-k8s-1.22/x86_64/latest/image_version",
"DataType": "text"
}
]
}
I've just found this issue (looking for another one :)) and decided to point eks-ng-ami-updater app (to have NG ami images up2date). Hope this is helpful while you wait for this feature 🙏 🤞 ...
