Proposal: weak bake dependencies

[jedevc]

Currently, dependencies between targets can be expressed as follows:

group "default" {
  targets = ["a", "b"]
}
target "a" {
  dockerfile = "a.Dockerfile"
}
target "b" {
  dockerfile = "b.Dockerfile"
  contexts = {
    ubuntu = "target:a"
  }
}

If the user runs docker buildx bake, then a is built first, then b is built using a instead of ubuntu as it's base image.

However, assume that the user runs docker buildx bake b to only build b - because b depends on a, a is also built. If the user has also set --push, then both a and b will be built, but only b will be pushed.

If a is not a perfectly reproducible build (e.g. it installs some packages which don't have fixed version numbers), then when b is pushed, it will have an entirely different set of layers than an a that was already pushed - sometimes this might not be desirable.

To resolve this, I propose weak references, i.e. references that will only be built if they would otherwise already be built.

e.g. target b could become (syntax tbd):

target "b" {
  dockerfile = "b.Dockerfile"
  contexts = {
    ubuntu = "weaktarget:a"
  }
}

If docker buildx bake b is run, then the ubuntu context should not be overwritten because a is not being built; conversely if docker buildx bake a b is run, then the context should be overwritten.

This proposal would ensure that for scenarios where not all targets are built at the same time, layer sharing can still be used and unnecessary builds don't need to be run.

[tonistiigi]

I'm not sure if a special syntax is worth it. The way I show this case in https://www.docker.com/blog/dockerfiles-now-support-multiple-build-contexts/ is a bake file written with:

target "a" {
  dockerfile = "a.Dockerfile"
}
target "b" {
  dockerfile = "b.Dockerfile"
}
target "b-stage" {
  inherits = ["b"]
  contexts = {
    ubuntu = "target:a"
  }
}

Now buildx bake b would build using the release dependencies and buildx bake b-stage with local dependencies(same as weak target with multiple input names in your case).

[jedevc]

I think that for a scenario of a separate debugging stage or a scenario like that, it makes sense to split them out and no new syntax is needed.

I think I'm more imagining a case where you might have a lot of targets, a, b, ..., z: in this case, the user might not know how the whole bake file fits together and so might not know to invoke the combination of targets to provide with -stage suffix. Also I think it gets much more confusing with chains of dependencies so that c -> b -> a.

R.e. syntax, it could be much more lightweight than what I'm suggesting - something like target:~a, or even a bake level flag like --weak-target-refs.

But agreed, I think it's quite a complex niche use case, I'm struggling to make the arguments that it deserves special syntax.