image-builder Add build info, such as image-builder version, as a file to built image

Is your feature request related to a problem? Please describe.

As part of https://github.com/kubernetes-sigs/cluster-api-provider-aws/issues/3107, while testing newly built images, we found out, that the images were built using older version of image-builder, which did not include some important patches.

It would be great if we could easily identify what version of image-builder was used to build the image. I could not find information or issue about such.

Describe the solution you'd like

It would be nice if built images could include information what version of image-builder were used for building this specific image. This is similar to #70, but I don't think version would have to be included in the image name. It could be a AMI resource tag or a special file on the root FS of the image with the information.

/kind feature

Feb 10 '22 22:02 invidian

Hi @invidian,

Would you be willing to take a second look at this? The code is in place to make this happen, and has been for quite some time. It should get a tag here.

For reference, I just checked a (very old) AMI that I had built, and I do see the image builder version in there as a tag:

Screen Shot 2022-03-03 at 2 38 33 PM

Mar 03 '22 21:03 codenrhoden

Hmm, it seems most recently published CAPA images does not have it:

$ aws ec2 describe-images --image-ids ami-046f5f86347c8d203 --region ap-northeast-1
{
    "Images": [
        {
            "Architecture": "x86_64",
            "CreationDate": "2022-02-18T08:45:21.000Z",
            "ImageId": "ami-046f5f86347c8d203",
            "ImageLocation": "258751437250/capa-ami-flatcar-stable-1.23.4-00-1645173023",
            "ImageType": "machine",
            "Public": true,
            "OwnerId": "258751437250",
            "PlatformDetails": "Linux/UNIX",
            "UsageOperation": "RunInstances",
            "State": "available",
            "BlockDeviceMappings": [
                {
                    "DeviceName": "/dev/xvda",
                    "Ebs": {
                        "DeleteOnTermination": true,
                        "Iops": 3000,
                        "SnapshotId": "snap-08b8908cc7c84dc7f",
                        "VolumeSize": 8,
                        "VolumeType": "gp3",
                        "Throughput": 125,
                        "Encrypted": false
                    }
                },
                {
                    "DeviceName": "/dev/xvdb",
                    "VirtualName": "ephemeral0"
                }
            ],
            "Description": "Cluster API base image designed for v1.23.4",
            "EnaSupport": true,
            "Hypervisor": "xen",
            "Name": "capa-ami-flatcar-stable-1.23.4-00-1645173023",
            "RootDeviceName": "/dev/xvda",
            "RootDeviceType": "ebs",
            "SriovNetSupport": "simple",
            "VirtualizationType": "hvm"
        }
    ]
}

Neither AMI or snapshot have no tags from what I see.

@sedefsavas do you have some idea why the tags are not present? Perhaps I should move this issue to CAPA repo?

Mar 03 '22 22:03 invidian

Cc @Ankitasw can this be because you were using the main branch?

Mar 03 '22 22:03 sedefsavas

The only reasons I know off the top of my head why it wouldn't show up are:

the machine you are building from doesn't have git installed (extremely unlikely)
you are building by calling packer instead of calling make

Mar 03 '22 23:03 codenrhoden

@Ankitasw @sedefsavas if you are available, let's chat about this next week. I'd like to get to the bottom of it, because taking a look again from the describe-images command that @invidian demonstrated, I see that none of the recently published CAPA images have any tags on them at all. This doesn't make any sense as tags are always applied to each build here: https://github.com/kubernetes-sigs/image-builder/blob/8b6dd4ae74f7bbf49941e590ab2dc1a048d67af9/images/capi/packer/ami/packer.json#L46-L57 It's not an optional config.

So I'd really like to see what's happening and how it's getting skipped. Right now it's a mystery to me! =)

Mar 04 '22 23:03 codenrhoden

Cc @Ankitasw can this be because you were using the main branch?

I am not sure if anything is changed in the main branch of image-builder.

The only reasons I know off the top of my head why it wouldn't show up are: the machine you are building from doesn't have git installed (extremely unlikely) you are building by calling packer instead of calling make

We have git installed in the machine, also I used make command to build the images. Even I am not sure what went wrong, only difference from last time we did is we used main branch to build images.

Mar 05 '22 05:03 Ankitasw

I am not sure if describe-images command is supposed to give the tags in output or not, but I can see the tags in all AMIs published in the AWS console of the account where AMIs are getting published. cc @codenrhoden @sedefsavas 157073533-51433650-cbc7-478c-8185-d78c7849980c

Mar 07 '22 16:03 Ankitasw

I don't think we can rely on describe-images output for tags as it doesn't give any tags info, I have confirmed the same for older images as well.

Mar 07 '22 16:03 Ankitasw

@Ankitasw do you know if tags are kept private to the user that built the image perhaps?

I built an AMI this weekend, and was able to see all the expected tags for my own AMI, but not for the AMI's published for CAPA. Are you able to see the tags via the AWS console perhaps?

Here's what I built:

$ aws ec2 describe-images --image-ids ami-0478758581673cf71 --region us-west-2 --output json
{
    "Images": [
        {
            "Architecture": "x86_64",
            "CreationDate": "2022-03-05T16:37:37.000Z",
            "ImageId": "ami-0478758581673cf71",
            "ImageLocation": "053033561758/capa-ami-amazon-2-1.21.10-00-1646497937",
            "ImageType": "machine",
            "Public": false,
            "OwnerId": "053033561758",
            "PlatformDetails": "Linux/UNIX",
            "UsageOperation": "RunInstances",
            "State": "available",
            "BlockDeviceMappings": [
                {
                    "DeviceName": "/dev/xvda",
                    "Ebs": {
                        "DeleteOnTermination": true,
                        "Iops": 3000,
                        "SnapshotId": "snap-088f5f889103dc31a",
                        "VolumeSize": 8,
                        "VolumeType": "gp3",
                        "Throughput": 125,
                        "Encrypted": false
                    }
                }
            ],
            "Description": "Cluster API base image designed for v1.21.10",
            "EnaSupport": true,
            "Hypervisor": "xen",
            "Name": "capa-ami-amazon-2-1.21.10-00-1646497937",
            "RootDeviceName": "/dev/xvda",
            "RootDeviceType": "ebs",
            "SriovNetSupport": "simple",
            "Tags": [
                {
                    "Key": "build_timestamp",
                    "Value": "1646497937"
                },
                {
                    "Key": "kubernetes_cni_version",
                    "Value": "v0.8.7"
                },
                {
                    "Key": "build_date",
                    "Value": "2022-03-05T16:32:18Z"
                },
                {
                    "Key": "kubernetes_version",
                    "Value": "v1.21.10"
                },
                {
                    "Key": "distribution_version",
                    "Value": "2"
                },
                {
                    "Key": "source_ami",
                    "Value": ""
                },
                {
                    "Key": "distribution",
                    "Value": "Amazon Linux"
                },
                {
                    "Key": "distribution_release",
                    "Value": "Amazon Linux 2"
                },
                {
                    "Key": "containerd_version",
                    "Value": "1.6.0"
                },
                {
                    "Key": "image_builder_version",
                    "Value": "v0.1.11-97-g8b6dd4ae7"
                }
            ],
            "VirtualizationType": "hvm"
        }
    ]
}

Mar 07 '22 18:03 codenrhoden

Are you able to see the tags via the AWS console perhaps?

Yes I am able to see tags in console as you can see in snapshot attached in previous comment.

@Ankitasw do you know if tags are kept private to the user that built the image perhaps?

I dont think tags are kept private to anyone, but atleast in any of old CAPA AMIs as well, I don't see any tags attached in the describe-images output.

Mar 07 '22 18:03 Ankitasw

Thanks @Ankitasw. Apologies that I missed your screenshot -- when I was scrolling through I thought it was mine. 🤦

It's really interesting that the tags show up in some cases and not in others. Right now I don't have an explanation for this.

Mar 07 '22 21:03 codenrhoden

Hmm, from https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Using_Tags.html#tag-restrictions:

When you tag public or shared resources, the tags you assign are available only to your AWS account; no other AWS account will have access to those tags. For tag-based access control to shared resources, each AWS account must assign its own set of tags to control access to the resource.

Mar 07 '22 22:03 invidian

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue or PR as fresh with /remove-lifecycle stale
Mark this issue or PR as rotten with /lifecycle rotten
Close this issue or PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

Jun 05 '22 23:06 k8s-triage-robot

/remove-lifecycle stale

This would still be super useful.

Jun 07 '22 09:06 invidian

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue or PR as fresh with /remove-lifecycle stale
Mark this issue or PR as rotten with /lifecycle rotten
Close this issue or PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

Sep 05 '22 10:09 k8s-triage-robot

/remove-lifecycle stale

Sep 05 '22 10:09 invidian

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue or PR as fresh with /remove-lifecycle stale
Mark this issue or PR as rotten with /lifecycle rotten
Close this issue or PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

Dec 04 '22 10:12 k8s-triage-robot

/remove-lifecycle stale

Dec 04 '22 11:12 invidian

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue as fresh with /remove-lifecycle stale
Close this issue with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

Mar 04 '23 12:03 k8s-triage-robot

/remove-lifecycle stale

Mar 04 '23 13:03 invidian

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue as fresh with /remove-lifecycle stale
Close this issue with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

Jun 02 '23 13:06 k8s-triage-robot

/remove-lifecycle stale

Jun 02 '23 14:06 invidian

@invidian I'm not sure what more image-builder can do here. The image-builder version is being published along with the AMI as an AWS tag but as you pointed out from the documentation, there's no way of accessing that for public images. For any images you build yourself, the image-builder version is there and can be accessed both via the web UI and via describe-images but that isn't possible for the CAPA provided AMIs.

The only other thing I can think of is if CAPA sets the ami_description to include the ib_version when they build their AMIs. (cc @Ankitasw @wyike)

Jun 19 '23 18:06 AverageMarcus

Do you mean add something more here https://github.com/kubernetes-sigs/image-builder/blob/master/images/capi/packer/ami/packer.json#L5 @AverageMarcus

Jun 20 '23 01:06 wyike

Specifically I mean https://github.com/kubernetes-sigs/image-builder/blob/b7f8d4c3a2cad49497ee29d2b409c2e708b9c137/images/capi/packer/ami/packer.json#L153

I'm not sure how you currently build the AMIs but if you're already providing a PACKER_VARS_FILE you can update it to include a new ami_description property with something like:

Cluster API base image designed for {{user `kubernetes_semver`}} built with image-builder {{user `ib_version`}}

Alternatively, the image-builder version could be included into the AMI name but that feels messy to me unless you plan to build the same Kubernetes versions against different image-builder versions.

Jun 20 '23 03:06 AverageMarcus

@invidian I'm not sure what more image-builder can do here.

For example, as I mentioned while opening this issue:

or a special file on the root FS of the image with the information.

Jun 21 '23 18:06 invidian

🤦 Sorry, I completely missed that while focussing on the tagging discussion.

Yeah, I think it makes sense to include a build-info file or something similar that includes various details about the built image.

I'm going to update the issue title to be more specific to that if you don't mind.

/retitle Add build info, such as image-builder version, as a file to built image

Jun 22 '23 07:06 AverageMarcus

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue as fresh with /remove-lifecycle stale
Close this issue with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

Jan 23 '24 04:01 k8s-triage-robot

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue as fresh with /remove-lifecycle rotten
Close this issue with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

Feb 22 '24 04:02 k8s-triage-robot

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Reopen this issue with /reopen
Mark this issue as fresh with /remove-lifecycle rotten
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

Mar 23 '24 05:03 k8s-triage-robot

image-builder image-builder copied to clipboard

Add build info, such as image-builder version, as a file to built image

image-builder
image-builder copied to clipboard