GitHub action for common tasks

[laurentsimon]

trafficstars

Across reusable workflows, we need:

• [x] Checkout the repo at the right ref (#49)
• [x] Build the builder
• [x] Download-verify the builder
• [ ] Declaration of common env variables (VERIFIER_BINARY, etc at the top of each workflow)
• [ ] Push assets to a release
• [ ] e2e and e2e pre-submit scripts (wrap their installation thru an Action)

We need common GitHub actions to perform these task, to avoid copy/paste of the same code.

The actions can be hosted in this repo and called by reusable workflows:

uses: slsa-framework/slsa-github-generator/actions/checkout-builder // Note: may just be hidden inside build-builder, I think
...
uses: slsa-framework/slsa-github-generator/actions/build-builder
  with:
    ref: needs.detect-env.outputs.ref
    repo: needs.detect-env.outputs.repo

...

uses: slsa-framework/slsa-github-generator/actions/download-builder
  with:
    name: ...
    sha256: ${{ needs.builder.outputs.sha256 }}

[asraa]

I LOVE THIS IDEA. This would make it foolproof to setup trusted builders. Which is more important than distributing builders :)

[ianlewis]

Is this achieving the same thing as #22 ?

[laurentsimon]

https://github.com/slsa-framework/slsa-github-generator/issues/22 suggests a provenance helper action to install the provenance CLI.

This issue suggests we create build helper actions.

Agreed, they are similar ideas.

[laurentsimon]

/cc @bcoe who is working on a PoC for the npm ecosystem. If you have further thoughts, let us know

[ianlewis]

@laurentsimon @MarkLodato I'm looking to move the detect-workflow action to a separate repo so it's easier to invoke in our workflows and we can avoid the chicken-and-egg problem we get when actions are in the same repo.

Are you able to create new repos under slsa-framework and could you create it for me? Since our reusable workflows will be the primary users I think it's best created here instead of another org. Maybe slsa-framework/detect-workflow-action?

[MarkLodato]

For my information, could you describe the chicken-and-egg problem? I would have thought that having it in the same repo is easier because then the whole workflow is hermetic.

[laurentsimon]

The action needs to be called by the re-usable workflow to determine at which ref it must checkout itself, but to call itself it needs to know a ref to itself. That's the chicken-and-egg problem. @ianlewis confirmed?

I think this means any other Action we create will go into this separate repo, e.g. the Action that compares hashes, the Action that upload assets, etc

[ianlewis]

The action needs to be called by the re-usable workflow to determine at which ref it must checkout itself, but to call itself it needs to know a ref to itself. That's the chicken-and-egg problem. @ianlewis confirmed?

Yeah, when you run an action in the same repo as the workflow, just like the workflow itself, you need to check out the repo beforehand. See: https://docs.github.com/en/actions/learn-github-actions/finding-and-customizing-actions#adding-an-action-from-the-same-repository

So in our case we could reference it like:

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      # This checks out the user workflow by default (listed in github context)
      # but we need the reusable workflow repo & ref.
      # So we need to specify the repo and ref here ourselves...
      - uses: actions/checkout@v3
      # ... but we don't know that until after this action runs!
      - uses: ./.github/actions/detect-workflow

But this won't work until we've checked out the repo already. We need to know the right ref in order to do that and the action is what's supposed to tell us the right ref!

Also, If we want to use this action in workflows outside of slsa-github-generator (e.g. slsa-github-generator-go, or slsa-github-generator-ko) or allow folk who make their own builders to use it, then it will need to be in it's own repo as well. This is because the syntax for referencing an action in a different repo require that repo to be dedicated to the action ({owner}/{repo}@{ref}).

Right now we have some inlined bash code in the yaml but it's ugly, doesn't do any verification of the OIDC token, and will need to be copied to any reusable workflow that needs to use it.

[MarkLodato]

Ah, thanks for the explanation! Could you put that in the new README for documentation?

I added you to the slsa-framework organization so you can create the repo yourself once you accept the invite. That name sounds good to me. Thanks!

[ianlewis]

After playing with this again it seems I was pretty mistaken. As @laurentsimon alluded to at the top, you can reference actions in another repo with a path so they don't need to be in a dedicated repo (Its not important but I had trouble doing this and misunderstood the reason why)

We just need to statically pick a good revision that we can use to run actions.

 - id: detect-workflow
   uses: slsa-framework/slsa-github-generator/.github/actions/detect-workflow@8d0cdc6af66f20f62b5a58df634077cb421b68fd
 - id: print
    shell: bash
    env:
      REPO: "${{ steps.detect-workflow.outputs.repository }}"
      REF: "${{ steps.detect-workflow.outputs.ref }}"
    run: |
      echo $REPO
      echo $REF

This is essentially what we would have to do if it was hosted externally in a different repo anyway. It's just potentially a bit more confusing for workflows in this repo because we're running actions at a previous version in this repo, while building with later versions of the same repo.

So I think an external repo is unnecessary. Sorry about the confusion.

[joshuagl]

I really like the idea of factoring out common pieces of the reusable workflows, however the GitHub documentation on Reusing workflows states, in the Limitations section that:

Reusable workflows can't call other reusable workflows.

[ianlewis]

I really like the idea of factoring out common pieces of the reusable workflows, however the GitHub documentation on Reusing workflows states, in the Limitations section that:

Reusable workflows can't call other reusable workflows.

Yeah, It's one reason we've so far gone with actions to hold reusable code, but it may become more of an issue as we have larger multi-step logic that needs to be reused.

[laurentsimon]

@naveensrinivasan the item Push assets to a release is the one where we re-write the upload ourselves, instead of relying on a 3P action. Worth asking whether this should be done in Go, TS or just gh CLI before starting the PR

[laurentsimon]

I'm going to take a stab at the artifact download/upload and builder action