[Core feature request] More sophisticated matrices

[Vampire]

What feature do you need? https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#example-using-contexts-to-create-matrices Currently only the most simple and flat matrices are supported with the datatype Map<String, List<String>>. But besides that you can take a matrix dimension value from a context variable, there are also much more advanced matrix use-cases that are not supported nicely yet.

For example with strategy: matrix: include: you can add entries to the matrix and with strategy: matrix: exclude: you can exclude some. And also a matrix dimension value does not have to be a flat list of strings, but it can also be a list of objects.

Do you have an example usage? Matrix from context variable:

matrix:
    version: ${{ github.event.client_payload.versions }}

Matrix with objects as values:

matrix:
    "environment":
        - "windows-2019"
        - "windows-2022"
        - "windows-latest"
    "distribution":
        - "wsl-id": "Debian"
          "user-id": "Debian"
          "match-pattern": "*Debian*"
          "default-absent-tool": "dos2unix"
        - "wsl-id": "Alpine"
          "user-id": "Alpine"
          "match-pattern": "*Alpine*"
          "default-absent-tool": "dos2unix"

Includes and Excludes:

matrix:
    "environment":
        - "windows-2019"
        - "windows-2022"
        - "windows-latest"
    "distribution":
        - "wsl-id": "Debian"
          "user-id": "Debian"
          "match-pattern": "*Debian*"
          "default-absent-tool": "dos2unix"
        - "wsl-id": "Alpine"
          "user-id": "Alpine"
          "match-pattern": "*Alpine*"
          "default-absent-tool": "dos2unix"
        - "wsl-id": "kali-linux"
          "user-id": "kali-linux"
          "match-pattern": "*Kali*"
          "default-absent-tool": "dos2unix"
        - "wsl-id": "openSUSE-Leap-15.2"
          "user-id": "openSUSE-Leap-15.2"
          "match-pattern": "*openSUSE*Leap*15.2*"
          "default-absent-tool": "which"
        - "wsl-id": "Ubuntu"
          "user-id": "Ubuntu-20.04"
          "match-pattern": "*Ubuntu*20.04*"
          "default-absent-tool": "dos2unix"
        - "wsl-id": "Ubuntu-18.04"
          "user-id": "Ubuntu-18.04"
          "match-pattern": "*Ubuntu*18.04*"
          "default-absent-tool": "dos2unix"
        - "wsl-id": "Ubuntu-16.04"
          "user-id": "Ubuntu-16.04"
          "match-pattern": "*Ubuntu*16.04*"
          "default-absent-tool": "dos2unix"
    "distribution2":
        - "wsl-id": "Debian"
          "user-id": "Debian"
          "match-pattern": "*Debian*"
          "default-absent-tool": "dos2unix"
    "exclude":
        - "environment": "windows-2019"
          "distribution":
              "wsl-id": "Debian"
              "user-id": "Debian"
              "match-pattern": "*Debian*"
              "default-absent-tool": "dos2unix"
          "distribution2":
              "wsl-id": "Debian"
              "user-id": "Debian"
              "match-pattern": "*Debian*"
              "default-absent-tool": "dos2unix"
        - "environment": "windows-2022"
          "distribution":
              "wsl-id": "Debian"
              "user-id": "Debian"
              "match-pattern": "*Debian*"
              "default-absent-tool": "dos2unix"
          "distribution2":
              "wsl-id": "Debian"
              "user-id": "Debian"
              "match-pattern": "*Debian*"
              "default-absent-tool": "dos2unix"
        - "environment": "windows-latest"
          "distribution":
              "wsl-id": "Debian"
              "user-id": "Debian"
              "match-pattern": "*Debian*"
              "default-absent-tool": "dos2unix"
          "distribution2":
              "wsl-id": "Debian"
              "user-id": "Debian"
              "match-pattern": "*Debian*"
              "default-absent-tool": "dos2unix"
    "include":
        - "environment": "windows-2019"
          "distribution":
              "wsl-id": "Debian"
              "user-id": "Debian"
              "match-pattern": "*Debian*"
              "default-absent-tool": "dos2unix"
          "distribution2":
              "wsl-id": "Ubuntu"
              "user-id": "Ubuntu-20.04"
              "match-pattern": "*Ubuntu*20.04*"
              "default-absent-tool": "dos2unix"
        - "environment": "windows-2022"
          "distribution":
              "wsl-id": "Debian"
              "user-id": "Debian"
              "match-pattern": "*Debian*"
              "default-absent-tool": "dos2unix"
          "distribution2":
              "wsl-id": "Ubuntu"
              "user-id": "Ubuntu-20.04"
              "match-pattern": "*Ubuntu*20.04*"
              "default-absent-tool": "dos2unix"
        - "environment": "windows-latest"
          "distribution":
              "wsl-id": "Debian"
              "user-id": "Debian"
              "match-pattern": "*Debian*"
              "default-absent-tool": "dos2unix"
          "distribution2":
              "wsl-id": "Ubuntu"
              "user-id": "Ubuntu-20.04"
              "match-pattern": "*Ubuntu*20.04*"
              "default-absent-tool": "dos2unix"

[Vampire]

Also null should be modelable, to for example model this which is not even possible right now on the one layer the matrices support:

strategy:
    fail-fast: false
    matrix:
        environment:
            - windows-2019
            - windows-2022
            - windows-latest
        distribution:
            - id: invalid
              label: invalid
            - id: ''
              label: ''
            - id: null
              label: 'null'

[jmfayard]

But isn't a strategy matrix just a for loop? How is that better than using a for loop in Kotlin? Apart from the resulting YAML being longer?

[krzema12]

@jmfayard nope, see related #287. Certain features of matrices are impossible to recreate with loops because they don't give enough hints to GitHub how to behave in case of e. g. failures. I agree, though, that for simple cases loops are enough.

[Vampire]

Not really a for loop, more a for loop over the combinations result. But it has slight differences like for example by default it fails fast if one job in the same matrix fails. And you can configure how many jobs in a matrix can run in parallel, for example if they use a common limited resource.

I also don't know whether there might be a maximum amount of jobs maybe that you could prevent using matrices? But that's just a wild guess.

And besides all that, for a migration I'd like to start with a script that produces the same result as before and then maybe refactor it to something different.

But the strongest arguments are the first two. :-)

[krzema12]

Moving this scope to #368.