Question: Automatically horizontally scale when building

[donifer]

Is this a request for help?: Yes

---

I'm trying to make my agents horizontally scale when all agents are busy. Lets say capping it at 10 workers and looks somewhat like this:

No pipelines triggered (agents => 1)
Incoming Work triggers pipelines
Agent_1 gets assigned and starts building
Automatically create Agent_2
Agent_2 gets assigned some work
Automatically create Agent_3
Agent_3 waits for job
Agent_1 finishes (gets killed)
Agent_2 finished (gets killed)
Remaining agents (1): Agent_3

[rimusz]

it is not supported by buildkite agent yet. Custom metrics need be involved there.

[nhooyr]

https://github.com/buildkite/elastic-ci-stack-for-aws already has this if you want it now

[nhooyr]

The metrics are collected via: https://github.com/buildkite/buildkite-agent-metrics

[rimusz]

I see Prometheus metrics are supported, which are cloud agnostic, which is good. Need to find time to play with it, at least somebody else can chip in with the help

[echoboomer]

@rimusz I realize this issue is old, but I opened this PR today to work on this: https://github.com/buildkite/charts/pull/80

As for whether or not is practical or useful, I'll leave it up to the team. Thanks!