Build step is confusing

[lox]

Having a build step without an image-repository step doesn't make a lot of sense. Based on discussion with @toolmantim it sounds like we should:

• Deprecate build
• Add a prebuild step that is what build was, but with image-repository mandatory
• Clarify in the docs how it should be used

[jgmchan]

I ran into this issue yesterday as I got the impression from the doco that the build step could save some time on a local build by prebuilding the image and utilizing the docker cache.

We don't have a docker registry to push to and only 1 agent so I just used the run step to do the building and the next steps can use the built image from the cache.

[lox]

@toolmantim just thinking out loud for the v3 syntax, what if we got rid of the implicit pull / push stuff and just made it more obvious when pull / push was happening?

It would make things so much less magical.

env:
  REGISTRY: "xxx.dkr.ecr.us-east-2.amazonaws.com/my-app"

steps:
  - label: ":docker: Prebuild"
    plugins:
      - docker-compose#v3.0.0:
          - build: 
              service: app
              cache-from:
                - ${REGISTRY}:latest-${BUILDKITE_BRANCH}
                - ${REGISTRY}:latest
          - push:
              images:
                - ${REGISTRY}:build-${BUILDKITE_BUILD_ID}
                - ${REGISTRY}:branch-${BUILDKITE_BRANCH}
                - ${REGISTRY}:latest

  - wait

  - label: "Tests"
    plugins:
      - docker-compose#v3.0.0:
          - pull:
              images:
                - ${REGISTRY}:build-${BUILDKITE_BUILD_ID}
          - run: 
              service: app
              image: ${REGISTRY}:build-${BUILDKITE_BUILD_ID}

[toolmantim]

I like the build and push in the first step, but then the pull and run is gonna be an epic amount of copy and paste config if you have 5+ different steps that use that same pre-built image (I’m thinking of buildkite/buildkite as an example)

[toolmantim]

I really like this though.

Reckon the “images” key is needed for push and pull? Can they be anything other than images?

The old syntax allowed you to configure a stack-level cache destination… with this one you’d need the env at the top of every pipeline.yml? 🤔 Maybe we could auto-pickup a repository env, that’s used as default for all push and pulls it doesn’t include a repo host in the path?

[lox]

I tend to think the copy and paste is better. It's more verbose, but there is less confusing magic. There are so, so many edge cases in the current setup.

[lox]

I mean, aren't you just copying and pasting image: ${REGISTRY}:build-${BUILDKITE_BUILD_ID} for each of your run steps? Is that really that bad?

[lox]

Perhaps we could automatically pull, so you don't need to have the explicit pull command.

[lox]

The old syntax allowed you to configure a stack-level cache destination

I kind of prefer it being explicit, it's confusing currently IMO.

[lox]

This is a longer example with the above changes:

env:
  REGISTRY: "xxx.dkr.ecr.us-east-2.amazonaws.com/my-app"

steps:
  - label: ":docker: Prebuild"
    plugins:
      - docker-compose#v3.0.0:
          - pull:
              images:
                - ${REGISTRY}:branch-${BUILDKITE_BRANCH}
                - ${REGISTRY}:latest
          - build: 
              service: app
              aliases:
                - app-with-mysql
                - app-with-postgres
              cache-from:
                - ${REGISTRY}:latest-${BUILDKITE_BRANCH}
                - ${REGISTRY}:latest
          - push:
              images:
                - ${REGISTRY}:build-${BUILDKITE_BUILD_ID}
                - ${REGISTRY}:branch-${BUILDKITE_BRANCH}
                - ${REGISTRY}:latest

  - wait

  - label: "Tests w/ MySQL"
    plugins:
      - docker-compose#v3.0.0:
          - run: 
              service: app-with-mysql
              image: ${REGISTRY}:build-${BUILDKITE_BUILD_ID}

  - label: "Tests w/ Postgres"
    plugins:
      - docker-compose#v3.0.0:
          - run: 
              service: app-with-postgres
              image: ${REGISTRY}:build-${BUILDKITE_BUILD_ID}

[lox]

Imagine if you could run commands in an image and save it 🤔

env:
  REGISTRY: "xxx.dkr.ecr.us-east-2.amazonaws.com/my-app"

steps:
  - label: ":docker: Prebuild"
    plugins:
      - docker-compose#v3.0.0:
          - build: 
              service: app              
              cache-from:
                - ${REGISTRY}:latest-${BUILDKITE_BRANCH}
                - ${REGISTRY}:latest
          - run:
               service: app
               command: [ "rake", "assets:precompile" ]
               commit: true
          - push:
              service: app
              images:
                - ${REGISTRY}:build-${BUILDKITE_BUILD_ID}
                - ${REGISTRY}:branch-${BUILDKITE_BRANCH}
                - ${REGISTRY}:latest

[lox]

I think I even prefer the explicit pull step before run:

env:
  REGISTRY: "xxx.dkr.ecr.us-east-2.amazonaws.com/my-app"

steps:
  - label: ":docker: Prebuild"
    plugins:
      - docker-compose#v3.0.0:
          - pull:
              images:
                - ${REGISTRY}:branch-${BUILDKITE_BRANCH}
                - ${REGISTRY}:latest
          - build: 
              service: app
              cache-from:
                - ${REGISTRY}:latest-${BUILDKITE_BRANCH}
                - ${REGISTRY}:latest
          - push:
              images:
                - ${REGISTRY}:build-${BUILDKITE_BUILD_ID}
                - ${REGISTRY}:branch-${BUILDKITE_BRANCH}
                - ${REGISTRY}:latest

  - wait

  - label: "Tests w/ MySQL"
    plugins:
      - docker-compose#v3.0.0:
          - pull:
              images:
                - ${REGISTRY}:build-${BUILDKITE_BUILD_ID}
          - run: 
              service: app-with-mysql
              image: ${REGISTRY}:build-${BUILDKITE_BUILD_ID}

  - label: "Tests w/ Postgres"
    plugins:
      - docker-compose#v3.0.0:
          - pull:
              images:
                - ${REGISTRY}:build-${BUILDKITE_BUILD_ID}
          - run: 
              service: app-with-postgres
              image: ${REGISTRY}:build-${BUILDKITE_BUILD_ID}

[lox]

We get to do away with all the complexity around building meta-data keys, and aliases, in exchange for being explicit about what is happening at a docker level.

[lox]

Reckon the “images” key is needed for push and pull? Can they be anything other than images?

Pull / push retries, and parameters to the pull or push command.

[lox]

That said, I think it probably simplifies things a lot to have push/pull just be a flat list.

[toolmantim]

Another take on the one above:

env:
  REGISTRY: "xxx.dkr.ecr.us-east-2.amazonaws.com/my-app"

steps:
  - label: ":docker: Prebuild"
    plugins:
      - docker-compose#v3.0.0:
          - build: 
              service: app
              cache-from:
                - “${REGISTRY}:latest-${BUILDKITE_BRANCH}”
                - “${REGISTRY}:latest”
          - push:
              - “${REGISTRY}:build-${BUILDKITE_BUILD_ID}”
              - “${REGISTRY}:branch-${BUILDKITE_BRANCH}”
              - “${REGISTRY}:latest”

  - wait

  - label: "Tests w/ MySQL"
    plugins:
      - docker-compose#v3.0.0:
          - run: 
              service: app-with-mysql
              image: “${REGISTRY}:build-${BUILDKITE_BUILD_ID}”

  - label: "Tests w/ Postgres"
    plugins:
      - docker-compose#v3.0.0:
          - run: 
              service: app-with-postgres
              image: “${REGISTRY}:build-${BUILDKITE_BUILD_ID}”

This just makes all the pulls for cache-from and run automatic.

Not having the push as part of the build feels weird tho, how does push know what service to push? What if you did multiple builds?

What about…

env:
  REGISTRY: "xxx.dkr.ecr.us-east-2.amazonaws.com/my-app"

steps:
  - label: ":docker: Prebuild"
    plugins:
      - docker-compose#v3.0.0:
          - build: 
              service: app
              cache-from:
                - “${REGISTRY}:latest-${BUILDKITE_BRANCH}”
                - “${REGISTRY}:latest”
              push:
                - “${REGISTRY}:build-${BUILDKITE_BUILD_ID}”
                - “${REGISTRY}:branch-${BUILDKITE_BRANCH}”
                - “${REGISTRY}:latest”

  - wait

  - label: "Tests w/ MySQL"
    plugins:
      - docker-compose#v3.0.0:
          - run: 
              service: app-with-mysql
              image: “${REGISTRY}:build-${BUILDKITE_BUILD_ID}”

  - label: "Tests w/ Postgres"
    plugins:
      - docker-compose#v3.0.0:
          - run: 
              service: app-with-postgres
              image: “${REGISTRY}:build-${BUILDKITE_BUILD_ID}”

[toolmantim]

Or…

env:
  REGISTRY: "xxx.dkr.ecr.us-east-2.amazonaws.com/my-app"
  IMAGE: “${REGISTRY}:build-${BUILDKITE_BUILD_ID}”

steps:
  - label: ":docker: Prebuild"
    plugins:
      - docker-compose#v3.0.0:
          - build: 
              service: app
              cache-from:
                - “${REGISTRY}:latest-${BUILDKITE_BRANCH}”
                - “${REGISTRY}:latest”
              push:
                - “${IMAGE}”
                - “${REGISTRY}:branch-${BUILDKITE_BRANCH}”
                - “${REGISTRY}:latest”

  - wait

  - label: "Tests w/ MySQL"
    plugins:
      - docker-compose#v3.0.0:
          - run: 
              service: app-with-mysql
              image: “${IMAGE}”

  - label: "Tests w/ Postgres"
    plugins:
      - docker-compose#v3.0.0:
          - run: 
              service: app-with-postgres
              image: “${IMAGE}”

[matthewd]

For reference, the existing config solves the "what are we pushing" bit with a prefix:

          - push:
              - "app:${REGISTRY}:build-${BUILDKITE_BUILD_ID}"
              - "app:${REGISTRY}:branch-${BUILDKITE_BRANCH}"
              - "app:${REGISTRY}:latest"

[toolmantim]

That last example still works without using aliases or meta-data!

I wonder if we could support both a map of build, run, push and pull… and an array? Which means this would also just work:

env:
  REGISTRY: "xxx.dkr.ecr.us-east-2.amazonaws.com/my-app"
  IMAGE: “${REGISTRY}:build-${BUILDKITE_BUILD_ID}”

steps:
  - label: ":docker: Prebuild"
    plugins:
      - docker-compose#v3.0.0:
          build: 
            service: app
            cache-from:
              - “${REGISTRY}:latest-${BUILDKITE_BRANCH}”
              - “${REGISTRY}:latest”
            push:
              - “${IMAGE}”
              - “${REGISTRY}:branch-${BUILDKITE_BRANCH}”
              - “${REGISTRY}:latest”

  - wait

  - label: "Tests w/ MySQL"
    plugins:
      - docker-compose#v3.0.0:
          run: 
            service: app-with-mysql
            image: “${IMAGE}”

  - label: "Tests w/ Postgres"
    plugins:
      - docker-compose#v3.0.0:
          run: 
            service: app-with-postgres
            image: “${IMAGE}”

[toote]

I believe that the discussions on this issue have shaped the way this plugin has evolved in the last few years and most - if not all - of what was discussed here is already possible so we should close this issue and review future changes based on what is currently possible in new issues. Any complaints?