containers-roadmap icon indicating copy to clipboard operation
containers-roadmap copied to clipboard

ECR image - protect specific version/tag from deletion

Open Space-Venom opened this issue 4 years ago • 7 comments

As a user of AWS container services, I would like to be able to "lock" a particular image version/tag and prevent it from being deleted either manually or via lifecycle policies.

Tell us about your request Add deletion protection to ECR repository images to prevent accidental deletion of important image versions when using automation, CLI or via lifecycle rules.

Which service(s) is this request for? ECR

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard? We push images to all environments regularly and tag as "latest" the image version we need to that environment when needed. Currently, lifecycle rules are set to keep 10 image versions, however, images are pushed several times per day. Sometimes, the "latest" tag image is being removed by the lifecycle policy. Both "image count" and "time since image pushed" policies can remove image versions that are important to the current environment.

Are you currently working around this issue? We will increase the number of images to 100 as a temporary solution and consider stop pushing images to all environments all the time and instead push only during CD, which will prolong the deployment time.

Space-Venom avatar Dec 08 '19 05:12 Space-Venom

From https://docs.aws.amazon.com/AmazonECR/latest/userguide/LifecyclePolicies.html

An image that matches the tagging requirements of a rule cannot be expired by a rule with a lower priority.

So, have the first rule in your lifecycle policy match tag 'latest' with eg. expire imageCountMoreThan 1. Then, any further rules will not expire your 'latest' image.

jsharper avatar Dec 08 '19 17:12 jsharper

Hi, we encounter a problem today. During the night one of our developer accidentally removed the prod.latest tag which is used by our ECS Task definition.

This morning traffic increased and when new Tasks tried to start they failed because our prod.latest tag was absent.

In our case our ECR Lifecycle policies are correctly configured and work since several months.

It would be great to add a feature in ECR to prevent human errors associated to critical Docker tags. In addition it would probably simplify ECR Lifecycle policies if the "lock" prevent Lifecycle policies to remove the tag.

Temporarily we'll develop a simple AWS Lambda function to be notified on our pager when a critical tag is accidentally removed.

bgaillard avatar Feb 12 '20 09:02 bgaillard

is there any news on this proposal ?

ilyesAj avatar Sep 07 '21 13:09 ilyesAj

Don't use mutable tags in production. Seriously, it's not a good idea, for the sole reason that it's much harder to guarantee what a mutable image tag resolved to at runtime, than when you are using an immutable tag.

All number of things, including pull policies, caching, etc can mean you even end up with 1/2 your Infrastructure running 2 different versions, even though they both report the same "tag".

ghostsquad avatar Dec 23 '21 06:12 ghostsquad

I came here because I was actually looking for a way to define an IAM policy that allowed me to specify specific tags or tag patterns as being mutable or immutable. But I've been unable to find any information about being able to control which images within a ECR repo can be managed via an IAM policy.

ghostsquad avatar Dec 23 '21 06:12 ghostsquad

I found this proposal looking for a way to avoid at least the last image of a repository to be deleted when a lifecycle policy is age based. For example, I have a project where we are tagging our images using semantic versioning and I would like to set a lifecycle policy to expire all images older than N days instead of the last X images. Maybe it would be a good idea to get a new "protect" action type available in adition to "expire" and a new countType called maybe "imageCountLessThan", so we could set a "protection" rule with lower priority and an "expiration" rule with higher priority to get this job done:

{
    "rules": [
        {
            "rulePriority": 1,
            "description": "Images Protection Rule",
            "selection": {
                "tagStatus": "tagged",
                "countType": "imageCountLessThan",
                "countNumber": 10
            },
            "action": {
                "type": "protect"
            }
        },
        {
            "rulePriority": 2,
            "description": "Default Image Retention Rule",
            "selection": {
                "tagStatus": "any",
                "countType": "sinceImagePushed",
                "countNumber": 30,
                "countUnit": "days"
            },
            "action": {
                "type": "expire"
            }
        }
    ]
}

hlalli avatar Aug 30 '22 19:08 hlalli

We have a WIP Image Retention Lambda that simply runs on a schedule. Some of the features of it are:

  1. The ability to define a "lineage" by regex, and you can control tags based on many different lineages within the same repo. An example of this:
myapp:409bca9b5599127a26f45dccf7a833d0651048ae

regex is [a-z0-9]{40}

these match what are essentially production images (git sha based). We only tag images produced after a merge main with these. Pull requests get:

myapp:pr-feat-123-409bca9b5599127a26f45dccf7a833d0651048ae

regex is pr-.+

We don't keep many copies of these. An improvement to the lambda would be to allow grouping based on a convention. That has yet to be implemented, such that, only the last 2 images for a given feature branch are kept, even if you have 20 active branches, you wouldn't end up with some branches having their images deleted.

ghostsquad avatar Sep 04 '22 22:09 ghostsquad