Support different git repository per pipeline step.

[buchgr]

Hi!

We are the team building Bazel (https://bazel.build). We want to have a pipeline that builds a Bazel binary at a specific commit, then takes this Bazel binary and checks out a different repository (i.e. Tensorflow) and tries to build and test it using this Bazel binary. Something like this:

steps:
  - label: "Build Bazel"
    command:
      - bazel build //src:bazel
      - buildkite-agent artifact upload bazel-bin/src/bazel

  - wait:

  - label: "Build TensorFlow"
    git_repository: https://github.com/tensorflow/tensorflow.git 
    command:
      - buildkite-agent artifact download bazel-bin/src/bazel /tmp/dir/ -step "Build Bazel"
      - /tmp/dir/bazel-bin/src/bazel build ...

We are currently working around this limitation by cloning TensorFlow in our own scripts and managing these repositories ourselves. However, it would be great if Buildkite could do that for us!

Please let me know what you think. Thanks!

[gtirloni]

Having just read something about this, maybe creating two pipelines and trigger the TensorFlow one (while passing some env var about where to locate the Bazel binary)? I haven't tested this so I'm not sure how (or if) they would share the build workspace/directory.

[phraemer]

Does anyone have any tips on how to achieve this?