Proposal: script directive for executing a script rather than calling run/build

[lox]

I find myself often writing scripts for calling docker-compose that pull the image from a previous build step. I'd love a way to automate some of the boiler plate of those. This is an example:

#!/bin/bash
set -eu

cleanup() {
  docker-compose kill || true
  docker-compose rm --force -v || true
  docker-compose down -v || true
}

trap cleanup EXIT

image_name=$(buildkite-agent meta-data get docker-compose-plugin-built-image-tag-blah)

export COMPOSE_PROJECT_NAME="buildkite$$" 
docker pull "$image_name"

... do docker-compose stuff

What if this could be:

steps:
  - command: test.sh
    plugins:
      docker-compose:
        script: my_custom_compose_script.sh

and the script could just be the docker-compose bits? Possibly even with a docker-compose-with-config function that calls the config files from the config.

Would save much pain, and allows extending on the plugin functionality without losing all of the nice bits.

[toolmantim]

I wonder if it should be called command?

And sourced, so it has access to the functions? That will make the functions a public API then, which is not necessarily a problem but we probably want to be clear if there are any that we don't want people to depend upon.

[lox]

Possibly, I guess that command sounds a lot like Docker's CMD.

I reckon if we did this we'd really limit what was exposed to the script, but yeah, it would be sourced I think.

[lox]

This has come up again, would be a nice way of calling custom run params that haven't been implemented yet. How about run-script, which would be a script called instead of the docker-compose run command. It would specifically override this line https://github.com/buildkite-plugins/docker-compose-buildkite-plugin/blob/3c1e578c636f7493cf65ac0c2302ec3eef6594e8/commands/run.sh#L141

  - command: test.sh
    plugins:
      docker-compose:
        run-script: my_custom_compose_script.sh

Then my_custom_compose_script.sh would look something like:

#!/bin/bash
set -euo pipefail

docker-compose -f docker-compose.yml -f "$BUILDKITE_DOCKER_COMPOSE_OVERRIDE" run --rm myservice

It would get invoked in a subshell with a specific set of ENV that it was passed from the parent plugin script, e.g BUILDKITE_DOCKER_COMPOSE_OVERRIDE with the override file generated.

It would be super "advanced users only", but would be a great way of customizing behaviour.

[toolmantim]

What’s the context/use-case that made this crop up again?

[toolmantim]

I haven’t checked, but I think you might still need the service name to ensure all the other bits work properly? Something like:

  - command: test.sh
    plugins:
      docker-compose:
        run: service
        command-override: my_custom_script.sh

This also means you can document it as a “run command only” option like the rest of them, rather than introducing another top-level key.

[lox]

@toolmantim discussion in https://github.com/buildkite-plugins/docker-compose-buildkite-plugin/issues/155 around whether we should do interpolation on $BLAH passed into various plugin arguments made me want a way to customize the run command.

[lox]

@toolmantim I reckon we'd need a separate one for build-script / build-command-override vs run-script / run-command-override.

I was thinking we'd set as much docker-compose env as we could so that invocation would be minimal, via https://docs.docker.com/compose/reference/envvars/ we could set COMPOSE_PROJECT_NAME and even COMPOSE_FILE.

[toolmantim]

Ugh, yeah #155 type issues pop up a bit now (wanting job-time env var interpolation, instead of pipeline-upload-time). I actually didn’t know about that no-interpolate on upload would cause it to be interpolated later.

Cool, well maybe something like this is the way to go?

[toote]

We are doing some cleanup on this repos' issues and found that what is being discussed here appears to actually be implemented in some way or another. Not sure if there is anything in particular that could be extracted off this conversation into another issue, though.