containers-roadmap
containers-roadmap copied to clipboard
aws
[ECR] [request]: add "sinceImagePulled" countType to ECR Lifecycle policy
Community Note
- Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
- Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
- If you are interested in working on this issue or have submitted a pull request, please leave a comment
Tell us about your request
Add a new sinceImagePulled countType to ECR Lifecycle policy.
Which service(s) is this request for? ECR
Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard? I would like to create an ECR Lifecycle policy that is based on when an image was last pulled. I can use such policy to infer that if an image has not been pulled in the last N months then it is not being used and I can safely delete it.
Are you currently working around this issue? Right now we have to carefully track which images are still in use by our various build and deployed systems. This is error prone and leads to accidental deletion of images that are still being used.
👍 Would love to see this getting prioritized (and am kinda surprised it's not already there!). 👍
I opened a new issue to publish metrics for a similar usecase, in case it's useful to anyone here: https://github.com/aws/containers-roadmap/issues/1587
Adding my voice to the chorus on this one. My team is pushing multi-arch images. Because of this, images are shown as untagged. Having a policy that simply deletes untagged images would be dangerous as I have no way if it is safe to delete. If I knew the image has not been pulled in the last N days would give us some confidence that we are deleting unused resources.
@sbkg0002 I see a metric for RepositoryPullCount in the docs for ECR private (released in January 2022), but not one that describes how recently an image was pulled. Can you give more detail of what you're seeing?
Hi all. We are tracking a lastRecordedPullTime but have not yet done work to integrate it into LCP. One concern we have is that while it's one piece of data, it doesn't necessarily indicate an image is safe to expire if it hasn't been pulled in some amount of time.
We were doing some work on a method to track whether or not a particular image is specified in a current deployment specification. As you may suspect that is a large bit of work and will take time to fully understand.
Given the upvotes on this issue and interest, we will pull it into consideration for our current round of planning. Thanks for the continued interest and input!
100% second this. We have thousands of images across hundreds of repositories. We provide the service to our production consumers. We need to remove images but careful as to not cause an outage where an image is in use.
We had to develop a custom solution to tag the images based on CloudTrail events. Pretty horrible workaround.
100% second this. We have thousands of images across hundreds of repositories. We provide the service to our production consumers. We need to remove images but careful as to not cause an outage where an image is in use.
We had to develop a custom solution to tag the images based on CloudTrail events. Pretty horrible workaround.
Us, too. But expecting AWS to do anything that saves their customers money is, well, not something I am holding my breath for.
@jlbutler
One concern we have is that while it's one piece of data, it doesn't necessarily indicate an image is safe to expire if it hasn't been pulled in some amount of time.
For us that wouldn't be an issue as we pull often. It covers many use cases so why not just release it while you build whatever more advanced feature you want to build?
We were doing some work on a method to track whether or not a particular image is specified in a current deployment specification.
That won't work for many use cases because not everyone that uses ECR has deployments inside AWS.
Hi @aviau
For us that wouldn't be an issue as we pull often. It covers many use cases so why not just release it while you build whatever more advanced feature you want to build?
For sure. I was just calling out a concern that I continue to have about leveraging this value on its own to indicate that an image is not in use. But as I indicated, we definitely are planning some work to improve the usefulness of the attribute (it currently tracks manifest pulls, which can make things confusing if you really want to know if the image was pulled including its layers), and we're looking at integrating this into LCP. We are still working on our roadmap, but we'll share more when there's something concrete.
That won't work for many use cases because not everyone that uses ECR has deployments inside AWS.
Yep it gets a bit tricky. Like you said, not everyone uses ECR on AWS, and not everyone using ECR deploys on AWS. The work I referred to may not be ECR-specific, potentially working for any image digest. Part of that could be opt-in solutions (e.g. a Kubernetes controller you can install to report image use). I don't think we can serve all use cases, but we're doing some research around that now, and don't have any formal features planned with it yet.
We'll post back here when we have a more concrete timeline for this LCP request, thanks again!
I can see this being a nice to have but I'd be careful in situations where underlying EKS nodes are caching container images for respective workloads
This feature would be extremely handy for my organization. In particular, we would like to implement this type of lifecycle on our pull-through-cache repositories.
this is very important feature to have. any update if anyone is looking into it?
Well, I wait this for years but I believe that features that really helps to save costs are not in priority at all :)
Hi all. We are tracking a
lastRecordedPullTimebut have not yet done work to integrate it into LCP. One concern we have is that while it's one piece of data, it doesn't necessarily indicate an image is safe to expire if it hasn't been pulled in some amount of time.
Doesn't "sinceImagePushed" (which is available in LCPs) suffer from the same shortcoming?
