docs icon indicating copy to clipboard operation
docs copied to clipboard

Provide example of using an output to define two matrixes

Open jsoref opened this issue 3 years ago • 23 comments
trafficstars

Code of Conduct

What article on docs.github.com is affected?

https://docs.github.com/en/actions/using-jobs/using-a-matrix-for-your-jobs#example-using-contexts-to-create-matrices

What part(s) of the article would you like to see updated?

Provide an example of using an output for two matrixes.

Often people will want to define a nontrivial matrix and then realize that they want to use it for more than one job and know that they don't want to copy and paste the definition because that's brittle. But they won't know how to share the definition for use by more than one matrix job.

Additional information

This question comes up often enough, e.g. here:

https://github.community/t/can-i-share-strategy-matrices-between-multiple-jobs/195827/7

Sadly, as you can see, people often get/give the wrong answer (no, you can't). Since it's definitely possible, and actually fairly easy, but not terribly obvious, it'd be helpful if it were documented.


[maintainers edit] See comments below for content creation ideas.

jsoref avatar May 10 '22 13:05 jsoref

Thanks for opening an issue! We've triaged this issue for technical review by a subject matter expert :eyes:

github-actions[bot] avatar May 10 '22 14:05 github-actions[bot]

This is a gentle bump for the docs team that this issue is waiting for technical review.

github-actions[bot] avatar May 17 '22 20:05 github-actions[bot]

That seems reasonable to me, that we would just include an example like this

juliandunn avatar Jun 09 '22 23:06 juliandunn

That fragment is a good start.

My preference would be to also include a sample createMatrices job to show setting the output with a matrix.

Thoughts?

jsoref avatar Jun 10 '22 00:06 jsoref

Sure. Do you have one handy that works?

juliandunn avatar Jun 10 '22 04:06 juliandunn

I don't think so offhand. It isn't a feature I use regularly, just something I've seen people ask for.

I could certainly find one next week.

jsoref avatar Jun 10 '22 11:06 jsoref

👋 @jsoref Just checking in on this one. Are you still up for providing the example? If not, let me know and I'll adjust some things on the board.

cmwilson21 avatar Sep 16 '22 20:09 cmwilson21

Hmm, I kinda wish I had left a breadcrumb. My current adventure involves setting runs-on and not matrix. I'm not sure offhand how to search my GitHub activity to find one.

I think it's best to ask someone else to work on this, I'm going to be very busy for the next month and really won't have time to work on this (or any of the other items I'm filing).

jsoref avatar Sep 19 '22 03:09 jsoref

Sounds great then, thanks for letting me know. 👍

cmwilson21 avatar Sep 19 '22 14:09 cmwilson21

Hey, I would like to work on this but I didn't quite get the type of example one would be looking for here.

notsosimppole avatar Oct 03 '22 03:10 notsosimppole

Can you teach me how to code

Sent from AT&T Yahoo Mail on Android

On Sun, Oct 2, 2022 at 10:55 PM, Gitansh @.***> wrote:

Hey, I would like to work on this but I didn't quite get the type of example one would be looking for here.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.Message ID: @.***>

YAKHATAAN avatar Oct 03 '22 04:10 YAKHATAAN

The general idea would be a dynamic matrix where one phase produces binaries and a second phase tests them.

jsoref avatar Oct 03 '22 04:10 jsoref

@cmwilson21 😦

jsoref avatar Aug 11 '23 18:08 jsoref

👋🏻 @juliandunn

We're reviewing all open help wanted issues as part of our preparation for Hacktoberfest next week.

Sadly the link you provided to a simple example in https://github.com/github/docs/issues/17748#issuecomment-1151724407 was lost when we migrated our community to discussions. Are you able to help with a new example?

felicitymay avatar Sep 19 '23 16:09 felicitymay

Thanks for opening an issue! We've triaged this issue for technical review by a subject matter expert :eyes:

github-actions[bot] avatar Sep 19 '23 16:09 github-actions[bot]

This is a gentle bump for the docs team that this issue is waiting for technical review.

github-actions[bot] avatar Oct 18 '23 16:10 github-actions[bot]

This is a gentle bump for the docs team that this issue is waiting for technical review.

github-actions[bot] avatar Nov 16 '23 16:11 github-actions[bot]

This is a gentle bump for the docs team that this issue is waiting for technical review.

github-actions[bot] avatar Jan 03 '24 16:01 github-actions[bot]

This is a gentle bump for the docs team that this issue is waiting for technical review.

github-actions[bot] avatar Feb 01 '24 16:02 github-actions[bot]

This workflow should be good enough:

name: shared matrix
on:
  push:
  workflow_dispatch:

jobs:
  define-matrix:
    runs-on: ubuntu-latest

    outputs:
      colors: ${{ steps.colors.outputs.colors }}

    steps:
      - name: Define Colors
        id: colors
        run: |
          echo 'colors=["red", "green", "blue"]' >> "$GITHUB_OUTPUT"

  produce-artifacts:
    runs-on: ubuntu-latest
    needs: define-matrix
    strategy:
      matrix:
        color: ${{ fromJSON(needs.define-matrix.outputs.colors) }}

    steps:
      - name: Define Color
        env:
          color: ${{ matrix.color }}
        run: |
          echo "$color" > color
      - name: Produce Artifact
        uses: actions/upload-artifact@v4
        with:
          name: ${{ matrix.color }}
          path: color

  consume-artifacts:
    runs-on: ubuntu-latest
    needs:
    - define-matrix
    - produce-artifacts
    strategy:
      matrix:
        color: ${{ fromJSON(needs.define-matrix.outputs.colors) }}

    steps:
    - name: Retrieve Artifact
      uses: actions/download-artifact@v4
      with:
        name: ${{ matrix.color }}
      
    - name: Report Color
      run: |
        cat color

Thoughts?

jsoref avatar Feb 27 '24 22:02 jsoref

@juliandunn Hello! 👋 I see that @felicitymay asked earlier in the timeline if you would be able to assist with the new example! Would you be able to review the one above that @jsoref has just posted?

nguyenalex836 avatar Feb 27 '24 23:02 nguyenalex836

name: shared matrix on: push: workflow_dispatch:

jobs: define-matrix: runs-on: ubuntu-latest

outputs:
  colors: ${{ steps.colors.outputs.colors }}

steps:
  - name: Define Colors
    id: colors
    run: |
      echo 'colors=["red", "green", "blue"]' >> "$GITHUB_OUTPUT"

produce-artifacts: runs-on: ubuntu-latest needs: define-matrix strategy: matrix: color: ${{ fromJSON(needs.define-matrix.outputs.colors) }}

steps:
  - name: Define Color
    env:
      color: ${{ matrix.color }}
    run: |
      echo "$color" > color
  - name: Produce Artifact
    uses: actions/upload-artifact@v4
    with:
      name: ${{ matrix.color }}
      path: color

consume-artifacts: runs-on: ubuntu-latest needs: - define-matrix - produce-artifacts strategy: matrix: color: ${{ fromJSON(needs.define-matrix.outputs.colors) }}

steps:
- name: Retrieve Artifact
  uses: actions/download-artifact@v4
  with:
    name: ${{ matrix.color }}
  
- name: Report Color
  run: |
    cat color

Luqman2302 avatar Feb 28 '24 01:02 Luqman2302

This is a gentle bump for the docs team that this issue is waiting for technical review.

github-actions[bot] avatar Mar 27 '24 16:03 github-actions[bot]

This is a gentle bump for the docs team that this issue is waiting for technical review.

github-actions[bot] avatar Jun 05 '24 16:06 github-actions[bot]

This example looks good to me @jsoref, thanks for writing it up.

I think that we could add a new example to this section that looks something like the following:

### Example: Using an output to define two matrices

You can use the output from one job to define matrices for the multiple jobs.

For example, the following workflow demonstrates how to define a matrix of values in one job, use that matrix in other jobs to produce artifacts, and then consume those artifacts in another job. Each artifact is associated with a value from the matrix.

<insert workflow .yml file here>

Notably, we have a couple reusables we can substitute into this workflow file instead of actually writing the artifact actions into the content.

{% data reusables.actions.action-download-artifact %}

{% data reusables.actions.action-upload-artifact %}

How does this look to you?

jc-clark avatar Jun 13 '24 18:06 jc-clark

Thanks. I've tried to apply that content -- I made a couple of changes to the references of "modifier job", but I think it's basically as you suggested. Let's see how it looks.

jsoref avatar Jun 13 '24 20:06 jsoref