Bake: allow inheriting from a map

[rster2002]

Description

Playing around with Docker Bake I found I needed to duplicate quite a bit of config as currently https://github.com/docker/buildx/issues/1203 is not yet supported. However, even with that feature implemented, it would be nice to inherit config from a map like, for example:

function "mkTarget" {
  params = [name]
  result = {
    tags = ["registry.example.com/${name}"]
    cache-from = [
      {
        type = "registry"
        ref = "registry.example.com/${name}:cache"
      }
    ]
    cache-to = [
      {
        type = "registry"
        ref = "registry.example.com/${name}:cache"
      }
    ]
  }
}

target "default" {
  inherits = [mkTarget("app")]
  context = "something"
  tags = ["registry.example.com/other:latest"]
}

This probably shouldn't use the inherits field, but this is just to give a general idea. It should probably not be possible to merge the contents of the inherits field in the map due to potential infinite loops.

I'd be happy to look into implementing this if this is something that would be useful.

[polarathene]

In the issue you referenced, I've already shown similar but you can use a matrix with your function to get something quite close, you just need to add the mappings (sometimes required such as for fields like tags if you want to merge instead of replace the fields from your mkTarget() function).

I adapted your function to use a for expression to wrap the object result, which provides support for returning an array of objects based on names inputs. That makes it a little bit more useful with matrix instead of calling the function per target name.

REGISTRY = "registry.example.com"

# Generate common fields for one or more targets:
function "mkTarget" {
  params = [names]
  result = [for name in names: {
    name = name
    tags = ["${REGISTRY}/${name}:latest"]
    cache-from = [
      {
        type = "registry"
        ref = "${REGISTRY}/${name}:cache"
      }
    ]
    cache-to = [
      {
        type = "registry"
        ref = "${REGISTRY}/${name}:cache"
      }
    ]
  }]
}

target "default" {
  matrix = {
    with = mkTarget(["app", "another-app"])
  }
  name = with.name

  context = "./docker/${with.name}/"

  cache-from = with.cache-from
  cache-to = with.cache-to
  tags = concat(
    with.tags,
    # Add an extra tag only for the "app" target:
    with.name == "app" ? ["${REGISTRY}/other:latest"] : []
  )
}

# Override or add specific fields to modify a target generated from the matrix above:
target "app" {
  dockerfile = "Dockerfile-app"
}

---

What you're asking for would still be a little nicer, as you can avoid the need to map to target fields like I've shown above. In a language like Javascript there is the spread operator ... that works similar to this, along with merge() function in HCL (similar to how inherits works under the hood AFAIK).

... is available in HCL, but only as the last function input, and only for lists IIRC. If it were possible to use with target it might look like this:

target "default" {
  matrix = {
    with = mkTarget(["app", "another-app"])
  }

  context = "./docker/${with.name}/"
  tags = concat(
    with.tags,
    # Add an extra tag only for the "app" target:
    with.name == "app" ? ["${REGISTRY}/other:latest"] : []
  )

  # Spread any object fields into this target that haven't already been defined:
  ...with
}

That's probably not a good UX though. You're basically requesting to be able produce a target via a function:

# Produces a target block (`target.from()` is a hypothetical method):
target.from(mkTarget("app"))

target "app" {
  # NOTE: Since the target name is the same, `inherits` would be redundant:
  #inherits = ["app"]

  # Add a new field:
  context = "something"

  # Replace existing tags field:
  tags = ["registry.example.com/other:latest"]
}

That's probably better than introducing a new field, other than the matrix use-case.

---

I found I needed to duplicate quite a bit of config as currently #1203 is not yet supported. However, even with that feature implemented, it would be nice to inherit config from a map

If that issue was resolved, I don't quite see the value you're proposing. You'd just use inherits with the templated target when all you need is the target name to be dynamic.

Other variations can be produced with logic and matrix without depending upon a function (even if the matrix only produces a single target. Then you just use inherits all the same.

[rster2002]

Setting the name was just an example here. I'm working with a lot of targets where I'd want to mix and match some parts of config while not impacting others. You can already do some of this by using the method described in the docs:

REGISTRY = "registry.example.com"

target "_cache" {
  cache-from = [
    {
      type = "registry"
      ref = "${REGISTRY}/${name}:cache"
    }
  ]
  cache-to = [
    {
      type = "registry"
      ref = "${REGISTRY}/${name}:cache"
    }
  ]
}

target "_versionTags" {
  tags = [
    "${REGISTRY}/${name}:${major}",
    "${REGISTRY}/${name}:${major}.${minor}",
    "${REGISTRY}/${name}:${major}.${minor}.${patch}",
  ]
}

target "default" {
  inherits = ["_cache", "_versionTags"]
}

But in this example even with the target name they would resolve to "_cache" and "_versionTags" (at least that's what I assume would be the case.) With this example you can of course split out the tags into a function that returns an array, but this is just to give an idea.

Using matrix doesn't seem that much better because I would still need to copy all the fields I'm interested in, instead of my function deciding what to set on its own. Something like this should also work:

REGISTRY = "registry.example.com"

function "mkVersionTags" {
  params = [base, major, minor, patch]
  tags = [
    "${base}:${major}",
    "${base}:${major}.${minor}",
    "${base}:${major}.${minor}.${patch}",
  ]
}

function "mkTarget" {
  params = [name]
  result = {
    cache-from = [
      {
        type = "registry"
        ref  = "${REGISTRY}/${name}:cache"
      }
    ]
    cache-to = [
      {
        type = "registry"
        ref  = "${REGISTRY}/${name}:cache"
      }
    ]
    tags = mkVersionTags("${REGISTRY}/${name}", 1, 2, 0)
  }
}

DEFAULTTARGET = mkTarget("default")

target "default" {
  cache-from = DEFAULTTARGET.cache-from
  cache-to   = DEFAULTTARGET.cache-to
  tags       = DEFAULTTARGET.tags
}

And then you'd need to copy the options for the default target and replace the variable. This already works, but it still requires a some boilerplate and more importantly if I want to set more options in the future, I'd have to visit all my targets and add them manually to each target.

[polarathene]

I'm a bit confused by your response 😕

Setting the name was just an example here. I'm working with a lot of targets where I'd want to mix and match some parts of config while not impacting others.

I understand that and I demonstrated that. I just adapted your example with a function and it's inputs.

But in this example even with the target name they would resolve to "_cache" and "_versionTags" (at least that's what I assume would be the case.)

It would not work there is no name variable or global attribute set. There is the proposed issue for something like target.get_name() and that would apply to the final target (after inherited).

Using matrix doesn't seem that much better because I would still need to copy all the fields I'm interested in, instead of my function deciding what to set on its own. Something like this should also work

Yes you could do that today, it's just using your function on a global scope attribute DEFAULTTARGET instead of setting via matrix (which is useful when you have multiple targets with slight differences such as target name).

This already works, but it still requires a some boilerplate and more importantly if I want to set more options in the future, I'd have to visit all my targets and add them manually to each target.

You only need to do that once, if you have multiple targets then use a matrix?

target "default" {
  matrix = {
    with = [mkTarget("app"), mkTargetAnotherWay("another-app")]
  }
  name = with.name
  context = "./docker/${with.name}/"

  cache-from = with.cache-from
  cache-to = with.cache-to
  tags = with.tags
}

# Any other changes specific to a target from above:
target "app" {
  dockerfile = "Dockerfile-app"

  # Replace the existing `tags` field:
  tags = ["registry.example.com/other:latest"]
}

The matrix makes it so you don't have to repeat yourself. Just have mkTarget() return an object of each field you'd map, with it's value as null if your internal customizations don't change that field sometimes, you can easily do this with the merge() function.

All you are asking for is to avoid mapping the fields explicitly (like tags = with.tags), which as shown above should only really need to be done once via a matrix base target, then just override any target as needed.