woodpecker
woodpecker copied to clipboard
Use multiple agents if matrix platform is used
Clear and concise description of the problem
When using the matrix system for building on multiple platforms as described in: https://woodpecker-ci.org/docs/usage/matrix-pipelines#example-matrix-pipeline-using-multiple-platforms Woodpecker will start building on one of the platforms and wait till that build is finished before it starts on the second platform.
The result is drastically increased overall build time.
Suggested solution
Run all platforms at the same time.
Alternative
No response
Additional context
In our case, if we were to fully switch to WoodpeckerCI, a PR CI build would take upwards of 5 hours if it would go through platforms one by one instead of running them all at once.
For clarification: If we define
matrix:
platform:
- 'linux/amd64'
- 'linux/arm64'
platform: ${platform}
I would want linux/amd64
and linux/arm64
to run at the same time.
Validations
- [X] Read the Contributing Guidelines.
- [X] Read the docs.
- [X] Check that there isn't already an issue that request the same feature to avoid creating a duplicate.
You have multiple agents? And what's your WOODPECKER_MAX_PROCS amount?
Yes, I have an agent on aarch64 and one on amd64.
My WOODPECKER_MAX_PROCS
is set to standard (which I assume is 1), since the documentation lists it as an "agent configuration options". My understanding is that this would enable one agent to build multiple pipelines simultaniously, but I want both my agents to work on one pipeline (since there is a matrix that requires both platforms)