Yelp
use events to do image cleanup correctly
We attempted this work in https://github.com/Yelp/docker-custodian/pull/28, but it fell through due to the fact that dockerd only saves the last 64 events, which is wholly insufficient for this purpose.
@dnephin: I think we really need this to get total correctness from dcgc. The only show-stopper is the persistence of docker events; there (essentially) is none.
Can we agree on a scheme wherein we dump the docker events to disk and have dcgc read it?
I imagine we simply define an init daemon like docker events > /var/log/docker/events.log and add an --events-log option to dcgc.
@bukzor that's what I proposed here: #28 (comment)
dcgc already supports a file based exclude list. Whatever writes the events to disk should be able to write just the image names.
I don't think --events-log makes sense in the context of dcgc. I could see it as a separate binary (in this package) that tails events and writes the file based on a set of rules. dciew (docker custodian image event watcher) ?
The logic to take an event stream and decide which images to exclude is what we added to dcgc in #28. I don't think pre-computing it in an external process makes sense.
The raw data is the event stream, and as such should be written to disk. Once we have this resource we enable other, unforeseen applications to make productive use of docker events, which is a Good Smell.
Also, if we get the list of events after we get the list of images, we eliminate the race wherein we delete a very recently pulled image because we have no events for it. Depending on an external process to do this computation opens this race back up imo.
I see what you're saying. It still feels a little weird to have dcgc read an event stream from a file, but I guess it's more reliable.
So I guess let's do it.
The file can be stdin in the case that you want to use docker events directly.