feedback
feedback copied to clipboard
buildkite
Better setup of interaction between pipelines
We're looking for a way to have interaction between pipelines for the following:
We have a build step that uploads an artefact then triggers a build in another pipeline. This is so that we can build part of our app asynchronously to the main app (the build is slow and doesn't need to block the main builld). The second pipeline then pulls down the artefact and uses it in its build.
This requires using the buildkite API for each of these interactions between the pipelines, and an access token for each. I understand that this is probably a bit of an edge case, but it would be nice to have some way of interacting between pipelines in the same org, without needing to setup access tokens, and to use buildkite's knowledge of both pipelines belonging to the same org to manage the access permissions for this.
Hey @watsonarw! Totally not an edge case, in fact, it's kind of what I see the future of Buildkite being all about :)
We've started making better tooling around this sort of setup already. Our first step was to allow builds to trigger other builds - and we've built that in now (but you've probably already seen this) https://github.com/buildkite/agent-tests/blob/master/tests/trigger-pipeline/.buildkite/pipeline.yml
The next step is to figure out cross-pipeline artifact downloads - which I'm very excited about. The hard bit is to figure out some permission systems around it.
But the goal is for you to do exactly what you described, without an API token in sight!
Has there been any progress on the artifact sharing part? We have the same use case and would love this feature.
Essentially in a first pipeline we want to build and test Bazel and upload the Bazel binary. We then want to trigger a few dozen pipelines that test downstream projects using this binary.
I'm in the same scenario. It would be great to be able to build something once and avoid re-building it down the line.
