charts
charts copied to clipboard
Question: Automatically horizontally scale when building
Is this a request for help?: Yes
I'm trying to make my agents horizontally scale when all agents are busy. Lets say capping it at 10 workers and looks somewhat like this:
No pipelines triggered (agents => 1)
Incoming Work triggers pipelines
Agent_1 gets assigned and starts building
Automatically create Agent_2
Agent_2 gets assigned some work
Automatically create Agent_3
Agent_3 waits for job
Agent_1 finishes (gets killed)
Agent_2 finished (gets killed)
Remaining agents (1): Agent_3
it is not supported by buildkite agent yet. Custom metrics need be involved there.
https://github.com/buildkite/elastic-ci-stack-for-aws already has this if you want it now
The metrics are collected via: https://github.com/buildkite/buildkite-agent-metrics
I see Prometheus metrics are supported, which are cloud agnostic, which is good. Need to find time to play with it, at least somebody else can chip in with the help
@rimusz I realize this issue is old, but I opened this PR today to work on this: https://github.com/buildkite/charts/pull/80
As for whether or not is practical or useful, I'll leave it up to the team. Thanks!