Allow to skip tasks defined in .cirrus.yaml using Starlark

[Jackenmen]

Description

I would want there to be a way to skip tasks defined in .cirrus.yaml from main() entry point of .cirrus.star. Ideally, it would be possible to alter more about a task than just whether it runs but I don't have a specific use case in mind for that just yet so I don't want to get this feature request rejected because I haven't come up with compelling use case for that :)

Context

I wanted to set up a Starlark main() entry point that would check the commit message for a custom trailer which I would use for defining a subset of tasks to run or a subset of tasks to skip but currently, there's no way to alter the tasks written in .cirrus.yaml. I don't want to move all my tasks over to Starlark as the YAML syntax is easier to digest as it's concise and unlike Starlark, it isn't a programming language.

Anything Else

Here's some example code of the functionality I wanted to implement (in main() entry point):

commit_msg = env["CIRRUS_CHANGE_MESSAGE"].strip()
trailers = []
for line in commit_msg.rsplit("\n\n", maxsplit=1)[1].splitlines():
    key, sep, value = line.partition(": ")
    if not (key and sep and value):
        trailers.clear()
        break
    trailers.append((key.strip(), value.strip()))


to_run = []
to_skip = []
for key, value in trailers:
    if key == "CI-run-task":
        to_run.append(value)
    elif key == "CI-skip-task":
        to_skip.append(value)

if to_run:
    to_run = [task_name for task_name in to_run if task_name not in to_skip]

    for task in all_tasks():  # some sort of built-in for working on tasks from YAML
        if task.name not in to_run:
            # if only skipping were implemented:
            task.skip()
            # if altering tasks were implemented fully:
            task.skip_if = True
else:
    for task in all_tasks():  # some sort of built-in for working on tasks from YAML
        if task.name in to_skip:
            # if only skipping were implemented:
            task.skip()
            # if altering tasks were implemented fully:
            task.skip_if = True

Example commit message:

Fix some Windows issue

CI-run-task: windows

[fkorotkov]

I think you can workaround the issue with existing functionality. Output of the Starlark script is appended to the YAML config and only then evaluated. That allows to override some thing thanks to https://github.com/cirruslabs/cirrus-cli/issues/398. So something like that might work in theory:

# .cirrus.yml

only_if: $TASKS_TO_RUN == '' || $TASKS_TO_RUN =~ '.*$CIRRUS_TASK_NAME.*'
skip: $TASKS_TO_SKIP =~ '.*$CIRRUS_TASK_NAME.*'

task:
  name: Windows
  ...

Then from your Starlark script you can return env with TASKS_TO_RUN and TASKS_TO_SKIP.

I'm actually not sure if $TASKS_TO_RUN =~ '.*$TASKS_TO_RUN =~ '.*.*'.*' will work:

1. If $CIRRUS_TASK_NAME is getting expanded
2. If =~ operator is case sensitive.

But if the above concept seems reasonable we can double check 1 and 2.

[Jackenmen]

This concept does seem reasonable, I would probably add some separator in the TASKS_TO_RUN/SKIP variable to avoid matching task_x_y when checking if Cirrus CI should run task_x, so e.g.

TASKS_TO_RUN=|task_x_y|windows|etc|

which I could then match with $TASKS_TO_RUN == '' || $TASKS_TO_RUN =~ .*\|$CIRRUS_TASK_NAME\|.*.

=~ should be case sensitive which seems fine for my use case I think but even if not, bash has ${CIRRUS_TASK_NAME,,} which would turn it into lowercase and I can make sure that TASKS_TO_RUN is lowercase from Starlark. I believe expansion for =~ works, though the single quotes might need to be dropped from the righthand side of the =~ operator. I guess I don't know how early Cirrus CI injects that var but I'm guessing it's probably going to be fine.

However, I'm unsure whether this will work for runners that don't use bash as a shell though (so e.g. Windows which uses cmd.exe, or alpine which uses base /bin/sh)?

[fkorotkov]

Our parser is written in Go so it's not runner related and should work regardless. 👌 Agreed on extra | for accuracy. BTW you'll nee quotes. I've just added an extra test case in https://github.com/cirruslabs/cirrus-cli/pull/529

[Jackenmen]

I have just tried this out and it doesn't seem to work: https://cirrus-ci.com/build/5104637312237568

I noticed that env from Starlark is appended to the end so I thought that maybe there's some evaluation order here that affects this so I also tried doing the same thing but in Starlark instead so that the env is before the only_if/skip but that didn't change anything. I suppose it's possible that the global only_if/skip is evaluated before CIRRUS_TASK_NAME is available though?

Edit: or maybe only_if/skip is only evaluated once for the whole build.

[fkorotkov]

Then I wonder of CIRRUS_TASK_NAME is available to boolevator. 🤔 Assigned to @edigaryev for further investigation.

[fkorotkov]

@edigaryev's fix should've fixed the issue. Plus take a look at #1053 for some easier combination of tasks from multiple YAML files.

[Jackenmen]

@fkorotkov sorry for the late reply but the solution proposed by you still does not work. Based on my testing, it appears that variables don't get interpolated into the regex strings, i.e. $CIRRUS_TASK_NAME here doesn't get replaced with the value of that variable:

only_if: $TASKS_TO_RUN == '' || $TASKS_TO_RUN =~ '.*\|$CIRRUS_TASK_NAME\|.*'

However, $CIRRUS_TASK_NAME is available from only_if and the variable gets properly included if it's not quoted, so something like this properly uses both $CIRRUS_TASK_NAME and $TASKS_TO_RUN:

only_if: $CIRRUS_TASK_NAME =~ $TASKS_TO_RUN

This means that I can just generate the actual regex string from Starlark and use that with =~ operator:

{
    "TASKS_TO_RUN": "|".join(to_run) if to_run else ".*",
    "TASKS_TO_SKIP": "|".join(to_skip) if to_skip else "$.",
}

only_if: $CIRRUS_TASK_NAME =~ $TASKS_TO_RUN
skip: $CIRRUS_TASK_NAME =~ $TASKS_TO_SKIP

My current problem is that this only works with only_if - the $CIRRUS_TASK_NAME variable is not available to skip. This can be reproduced easily by putting a simple skip: $CIRRUS_TASK_NAME =~ '.+' line and seeing that it causes no tasks to be skipped. For comparison skip: $CIRRUS_TASK_NAME =~ '.*' will cause all tasks to be skipped since a non-existent $CIRRUS_TASK_NAME variable gets interpreted as an empty string.

[Jackenmen]

@fkorotkov should I make a new issue for what I described in the previous comment?