docker
Export build cache even if build fails
Hello,
I would like to propose as a feature that even if a build fails, Buildx should still export caches specified with --cache-to.
Rationale
We have a few lengthy build steps in our project (which are run in CI) that can usually be reused from cache, but if you have to change something that re-triggers some of those steps and you get it wrong such that any but the first step fail, you're back to waiting for those previously successful steps because they are not cached.
It is my understanding, through observed behavior, that Buildx will only export caches at the end of a successful build. Having an option to make it export the cache for successful layers even if the overall build failed would allow us to reuse lengthy build steps that succeeded in a failed build.
There is a workaround by dividing lengthy build steps into individual stages and then running these stages in order, each of them caching to and from the same location. But this becomes increasingly difficult to manage as the build complexity increases, with each lengthy build step not only requiring its own Docker layer but also its own job/invocation.
I'm not quite clear on the exact behavior here but I want this all to work more!
if I use --cache-to type=local,dest=/docker_cache/blah,mode=max --cache-from type=local,src=/docker_cache/blah I get pretty good behavior. I'm not 100% certain of the behavior if there is a failure the first time the image is built (with no blah directory yet existing) -- but on subsequent executions, if there is a failure near the end of the Dockerfile execution -- the previously run steps tend to be in the cache when expected.
I'm pretty sure this is not the case with the --cache-to type=registry. It would be very nice to export to the registry cache even when build fails
For what it is worth: podman build --cache-to="..." will push to remote registry cache on each successful build stage completion (making it independent of whether the end result is success or failure).
Just note the syntax differences:
# Docker CLI arg:
docker build build ... --cache-to="mode=max,image-manifest=true,type=registry,ref=my.registry.io/my-image:cache"
# Podman CLI arg
# (no tag or extra parameters allowed as it is required for docker)
podman build ... --cache-to="my.registry.io/my-image/cache"
