agent-stack-k8s
agent-stack-k8s copied to clipboard
[WIP] Pre-schedule hook
WIP: This currently doesn't work. The controller container simply doesn't have permissions to execute a script. (That sounds sensible actually?) So it will have to go in another container.
The stack can configure the agent to run hooks easily. In fact the agent container ships with a config file that sets the hook path to /buildkite/hooks
, so one merely need mount some hooks at that path and the agent will run them, including pre-bootstrap
.
However, with podSpecPatch
, a pipeline can trivially override the hooks path (e.g. setting a BUILDKITE_HOOKS_PATH
env var) which would disable pre-bootstrap
being run. So pre-bootstrap
no longer works as a mechanism to check jobs before running.
This PR adds a new special hook for the k8s controller: pre-schedule
. Similar to pre-bootstrap
it has the ability to inspect the job the controller is about to schedule a pod for. Rather than "env var" form, I've opted to dump the whole job as JSON.
Since this happens in the scheduler, I've gone ahead with a long-desired refactor of some of the scheduler methods to eliminate the err
struct field.