woodpecker
woodpecker copied to clipboard
woodpecker-ci
Ability to limit concurrent pipelines
Clear and concise description of the problem
I remember that original CI had a feature called concurrency https://discourse.drone.io/t/how-to-limit-build-concurrency-per-project/3500 This helped a lot to prevent build run when other build already in progress. If you commit faster then old build ends new build starts and sometimes they need to spawn e.g. 2 services which have conflicts. Is there an option to somehow limit build to 1 at a time? I know it could be resolved if we would use 1 agents, but then other projects on same CI instance would stuck.
Suggested solution
Adding option or any other solution which is not obvious from current doc.
Alternative
No response
Additional context
No response
Validations
- [X] Read the Contributing Guidelines.
- [X] Read the docs.
- [X] Check that there isn't already an issue that request the same feature to avoid creating a duplicate.
This would also be great if there was not only a "users choice" but also an "admin only / server config" way to define it, which would allow for a more equal resource allocation if multiple projects run on the same CI (think Codeberg-CI as an example).
Especially when the pipelines running don't reach a projects' resource quota (feature TBD) but flood the agents with slow tasks (high IO-load comes to my mind), limiting to X per project would even out the field a bit as else running hundreds of pipelines from one project will slow down all others (let them wait for a free agent).
And the order is important, the pipelines should run in FIFO order. Otherwise even if the concurrency limit is 1 but the pipelines run out-of-order, the old result might overwrite the new one.
And the order is important, the pipelines should run in FIFO order. Otherwise even if the concurrency limit is 1 but the pipelines run out-of-order, the old result might overwrite the new one.
Agree, very important point, especially assuming time entropy caused by docker builder cache and so on
@6543 @anbraten Are there any potential features (implemented or in roadmap) that could replace this feature? We are still catching the consequences of this issue from time to time on various production projects.
Even if achieving builds queue (I mean one queue for unique repo and branch) is impossible, Probably there is at least a simple way to prevent start of a new build when previous build on the same branch of same repo was not yet finished.
I think there was still no direct way added to achieve this yet. You could use the / an adjusted autoscaler to spawn agents for each repo. I just updated some ideas for limits in https://github.com/woodpecker-ci/woodpecker/discussions/2303#discussioncomment-8117424. You are requesting a concurrent limit per repo and not per org right?
You are requesting a concurrent limit per repo and not per org right?
Yes per repo. The issue is when old build in progress and someone commits, woodpecker starts another build instantly
Yes per repo. The issue is when old build in progress and someone commits, woodpecker starts another build instantly
Yip. In most repos I worked with we enabled the skip if new pipeline is started which solves this somehow, but doesn't help if you require all pipelines to run.
Yes per repo. The issue is when old build in progress and someone commits, woodpecker starts another build instantly
Yip. In most repos I worked with we enabled the
skip if new pipeline is startedwhich solves this somehow, but doesn't help if you require all pipelines to run.
Yep, exactly, then there is a risk that latest commit might not be deployed. Generally at software development company we have a lot of projects (startups, playgrounds, MVPs, PoCs) for every client so binding to agents might be a little bit more complex to achieve. Development time is precious for all clients so we generally just push and pass to QAs the deployed env, or check by ourselves and then if problem appears that state is not deployed we investigate why (to minimize time spent)
But I still have a feeling that we are doing something wrong because a lot of woodpecker users somehow exist without requirement to limit build concurrency and this GitHub issue is not so popular as I thought it might be. Or probably vast majoroty is using only one build agent so there is a natural concurrency limit. Or build queue is implemented manually somehow. Or others simply ignore overlaps which are not so often if not so many developers work on repo
But I still have a feeling that we are doing something wrong
You are not alone. We use Ansible for deployments and Ansible Molecule for automated testing of roles and for those tasks I need to ensure that the deployment and test workflow are executed in the correct order and not in parallel, skipped or cancelled.
You are requesting a concurrent limit per repo and not per org right?
In Drone concurrency can be set per pipeline not per repo and I would like to have the same ability in Woodpecker. In my case per repo would work as well but as e.g. linting and other none-critical tasks can still be executed in parallel I would prefer a per workflow config instead.
