Not finding all updates

[mhbates]

DockCheck seems to not find all updates, and in fact finds a different update that other tools don't find. For reference, I have containers running via Portainer stacks (which uses Compose).

Watchtower reports that a certain set of containers have newer images (which is accurate), while DockCheck doesn't report any of those, but reports that a different container has an update that Watchtower. Pretty strange.

[mag37]

Thank you for reporting.

I've had issues before with how Portainer builds the container labels and running configs - fields don't look the same as they do running plain docker compose and the image check logic breaks.

Will try to troubleshoot later, but what do you get if you try this on suspected containers:

docker inspect "ContainerNameHere" --format='{{.Config.Image}}'

Then with the previous output run this: docker image inspect "ConfigImageHere" --format '{{.RepoDigests}}'

And paste both outputs here if you don't mind.

[mhbates]

So, for the first command, it outputs as expected - the same image that is used in the compose. Image:latest.

The second command throws an error saying that there is no such image.

I have noticed something that might relate to this. Basically, outdated images no longer show their tags in Portainer, but rather the hash; and I'm guessing this happens at the docker level as well based on the error the second command threw.

It looks like this is a Portainer issue. I'm going to do some additional research and see if I can figure it out. I'll report back.

[mhbates]

As an aside, I had set up a script to prune docker images that were no longer in use. So, once those image names were hashed, the "latest" images were then pruned/deleted by the script overnight, and running DockCheck now does find the containers needing updates. So, even if I can't figure out what is causing the hashing of the name, I could still use DockCheck and just wait a day for my prune script to run :)

[mag37]

That's some odd stuff - that they loose their tags and that the image-hash cant be found. I'll paste my output just to show how it usually looks, if that helps with your troubleshooting.

$ docker inspect "npm" --format='{{.Config.Image}}'
jc21/nginx-proxy-manager:latest
$ docker image inspect "jc21/nginx-proxy-manager:latest"  --format '{{.RepoDigests}}'
[jc21/nginx-proxy-manager@sha256:e5407dfe0577301171f403ec6b8a01b87eca5420416a907f9c42775428e22fee]

I'm not really following what you mean by hashing of the name - but it sounds like you're pruning images currently running too? If you restart a container after you prune, does it pull the image again?

I'll try to spin up portainer and do some testing when I got time. Sorry I cant help you more right now.

[mhbates]

What happens is that the image name/tag is renamed to just the hash of the image, rather than the 'jc21/nginx-proxy-manager:latest', for example.

I only prune images that are no longer in use. So, my theory is that it has something to do with Watchtower or Portainer, and one of those changing the image name. For example, what might be happening is that Watchtower pulls the new image to compare it to the one currently in use, and if they do not match, it might keep that new image on-hand, and rename the old one to just its hash. Something like that.

I'm going to keep testing it over the next couple days. My first test case is to not use Watchtower at all, and only check for updates using DockCheck. That might be the way to go. I'll circle back on this.

[FjellOverflow]

I actually have a very similar issue. I'm running Watchtower to solely notify me of new images, which it does, and when running dockcheck, I get different/missing update suggestions.

I can see that with Watchtower only notifying, not updating, it seems to still pull and hence retag the images, as I do have new images tagged with latest on my machine and the old latest are untagged and my container is suddenly displayed to run image 468bfef51db2 rather than audiobookshelf:latest.

I've never created any container through Portainer, so I'm fairly sure this issue is just Watchtower pulling &retagging images, and dockcheck running after not finding updates for containers running on the now untagged images.

[mag37]

@FjellOverflow

I can see that with Watchtower only notifying, not updating, it seems to still pull and hence retag the images, as I do have new images tagged with latest on my machine and the old latest are untagged and my container is suddenly displayed to run image 468bfef51db2 rather than audiobookshelf:latest.

I've not tested with Watchtower in ages - so cant really speak about the current functionality behind it. But the reason why this project (Dockcheck) was born is because the community was saying "Why does Watchtower have to pull all the images - isn't there a way to just check for diffs?" and there wasn't anything at the time so I made a small proof of concept which grew to this.

So you're most likely correct in your assessment - it still pulls the images. Which I think is odd, because what if you didnt want to update? If you'd restart the container for other reasons and you now have (due to Watchtower pulling) the latest image locally - the container would now have started with the new image.

I'm not saying you shouldn't use Watchtower - it's still a great project, just different ways of handling things. And if you'd like notifications you could try set up one of the available ones in this project :)

[FjellOverflow]

So you're most likely correct in your assessment - it still pulls the images. Which I think is odd, because what if you didnt want to update? If you'd restart the container for other reasons and you now have (due to Watchtower pulling) the latest image locally - the container would now have started with the new image.

I do agree! It is kind of problematic behavior on Watchtowers side, I just assume most people actually do want it to also update their containers, so only few people are affected.

I'm not saying you shouldn't use Watchtower - it's still a great project, just different ways of handling things. And if you'd like notifications you could try set up one of the available ones in this project :)

Also agree here :) There seems little reason to run both tools side-by-side, I just encountered the issue and thought I'd report the findings and that it's most likely purely a Watchtower issue and dockchecks behavior is just as expected. Maybe a small warning somewhere about problematic interaction with Watchtower could give people a heads-up, but you can probably close this issue, as its entirely on Watchtowers side :)

[mag37]

Indeed! A warning will be added to the readme as this might be a headscratcher from some random/odd behavior for some.

And I'm grateful you reported, shed some light on what might be causing it. And while we were digging at Portainer in this threads history, the issue might have been caused entirely (or mostly) by running Watchtower alongside Dockcheck.

Thank you, I'll close it when I add the note to readme later!