action-validator
action-validator copied to clipboard
Ability to restrict runs-on
Intro
Hey, following recent changes, essentially any value of runs-on:
is accepted. Since one of the outcomes of this action is to reduce the possibility of broken workflows, I think it would be very beneficial to validate runs-on somehow.
Proposal
- By default validate against freely available GitHub Hosted runners (no XL, no self-hosted)
This way the validator brings immediate value to hopefully most users, - With a CLI parameter (
--allowed-runners
) allow setting valid (incl. custom) values.
but also gives the ability to override if needed.
Syntax
- Comma separated list
- Prefix
@
for built-in (in the validator) groups - Prefix
-
for exclusion - No
-
prefix = positive listing
Built-in groups
- github-free: ubuntu-latest, ubuntu-22.04, etc.
- github-latest: ubuntu-latest, windows-latest, etc.
- github-xl: -xl and -cores suffixed ones
- self-hosted: special value for current 0.5.4 behaviour
Custom groups
If there's a known list of self hosted runners, it's possible to just list them all and use no groups. That is, to define a custom group, users need to list all runners explicitly.
For reusability the custom groups could be extracted into environment variables / organization level variables, like ACTION_VALIDATOR_ALLOWED_RUNNERS
, and then that used in CLI with shell expansion: --allowed-runners=${ACTION_VALIDATOR_ALLOWED_RUNNERS},-ubuntu-latest
.
Calculation
- Start with empty set
- Add all positive groups, and positive runners
- Remove all negative groups and runners
i.e. order of flags doesn't matter.
Examples
- the default when no argument provided
--allowed-runners=!github-free
- a GitHub Enterprise user might validate like this, if they allow 2 self-hosted runners, and want to be explicit about versions:
--allowed-runners=!github-free,@github-large,-@github-latest,my-special-runner1,my-special-runner2
- an open-source community member GitHub user, who wants to use latest without automagic updates:
--allowed-runners=ubuntu-22.04,windows-2022,macos-12
Hi Robert, thanks for starting this convo.
I'm definitely in favour of introducing a way to better validate runs-on
values. Not being a custom runner user myself, I'm probably not in a strong position to define a specification for how to do it. Thus, I can at best kibbitz on the work of others.
The only feedback I've got on your proposal so far is that I'm wary of using !
in a command-line argument, as shells have a habit of treating that character specially. I know of programs that use @
as a prefix for group-like behaviour; would that be an acceptable prefix in this case?
Thanks. I agree with @
, changed above and reformatted a bit.
I'm not a self-hosted user either, so let's get opinions from those who do: @MPV (#59), @mathew-fleisch (#5), @deviantintegral (#51)
👏 This all looks good to me. It gives us a way in CI to make sure runners are valid, catching typos in custom runner names.