act

act copied to clipboard

My use case: I would like to be able to use act to be the runner for a set of unit tests around my Actions workflow - preferably without having to modify my workflow file. One step towards this goal would be to be able to specify custom stubs for jobs, steps in jobs, or specific actions. In this manner I can selectively override the outputs of those portions and thus be able to properly skip portions of the workflow or inject data, depending on my tests' needs.

I will post my suggestion for accomplishing this in the comments as it might not be the best design or has other issues - in my opinion the feature request and the implementation details should always be separate: the former describes a need, the latter describes one possible way to accomplish the need.

One possible implementation would be to add a CLI flag such as -o --overrides folderpath for a folder of YAML files structured as follows:

name: An example overrides file # REQUIRED string. Same as GitHub Actions Workflow name clause.

on: # OPTIONAL object. Same rules as GitHub Actions Workflow on clause. If not specified then apply to ALL events.
  workflow_call:

regex: # OPTIONAL object. Sets rules for for any RegEx matches in the below.
  engine: "javascript" # OPTIONAL string. Sets which RegEx parser ruleset will be followed. Pick one, make it the default - probably whatever is Go's default. Over time the community can add more and use this to allow the user to choose.
  flags: "i" # OPTIONAL string. Sets the flags for any RegEx matches in the below. Default should be whatever the chosen engine provides as default.

overrides: # REQUIRED object.
  jobs: # OPTIONAL object. Each key is a RegEx that matches against job IDs. More than one hit is acceptable.
    "^name.*-regex": # object. This variant has a "run" clause and thus replaces ALL the steps of the matching jobs.
      run: myShellScript # REQUIRED string. Similar to the GitHub Actions Workflow run clause. Script would execute for every matching job. Path is relative to this override YAML file.
      shell: bash # OPTIONAL string. Same as GitHub Actions Workflow shell clause.

    "^jobname": # object. The variant has a "steps" clause that causes the matching steps in the matching jobs to be replaced .
      steps: # REQUIRED object. Defines per-step overrides.
        "debugging handle": # REQUIRED object. Just a name to reference this entry by. Has no semantic meaning.  If no Matcher entry is defined in the below clauses then this override applies to ALL steps.
          run: # REQUIRES string. Similar to the GitHub Actions Workflow run clause. Script would execute for every matching step per matching job. Path is relative to this override YAML file.
          shell: bash # OPTIONAL string. Same as GitHub Actions Workflow shell clause.
          name: # OPTIONAL string. Matcher. Glob that matches steps that have a name entry.
          id: # OPTIONAL string. Matcher. Glob that matches steps that have an id entry.
          uses: # OPTIONAL string. Matcher. Glob that matches steps that have a uses entry.

    #... more job overrides

Any and all shell scripts called or written in the above execute in the workflow runner as if they were steps in the matching job. Thus shell scripts can define outputs and logging using the same utilities as a step, e.g. ::set-output etc.

If the script is executing as a job, then the following JSON object is passed to the script on STDIN:

{
  "matrix": // Object. The data from the `matrix` object for the matching job.
  "name": // String. Name of the job.
  "needs": // Object. The data from the `needs` object for the matching job.
  // Am I missing anything useful?
}

If the script is executing as a step, then the following JSON object is passed to the script on STDIN:

{
  "id": // String | undefined. ID of the matching step.
  "name": // String | undefined. Name of the matching step.
  "run": // String | undefined. `run` clause of the matching step.
  "script": // String | undefined. `script` clause of the matching step.
  "shell": // String | undefined. `shell` clause of the matching step.
  "uses": // Object | undefined. The data from the `uses` object for the matching step.
  "with": // Object | undefined. The data from the `with` object for the matching step.
  // Am I missing anything useful?
}

If any two or more matchers hit the same job, then the job is run under all such rules in combination. For example if a job in the workflow is given the ID test and the overrides had the following job matchers: "*" and "test" then the overrides for "*" would be applied AND the overrides for "test" would be applied. The output of such a job should be the same as if multiple steps in a job set the outputs: last I checked that rule was "last write wins" - but may be undefined in the spec.

The rules for when multiple override steps match the same workflow step: the scripts execute in an undefined order, last write to the output wins.

Expressions are allowed, following the same rules as GitHub Actions Workflows. Determining which if any contexts, functions, and variables from Actions are allowed in the overrides YAML document is outside my scope ATM. However the following additional contexts are recommended:

act:
  cli_flags:
    overrides: # string. The path provided by the CLI flag.
  override:
    file: # string. Path to the current override YAML file.
    target: # string. URL for the exact path and overridden entry. e.g. `file:///home/user/demo/.github/workflows/ci.yaml#jobs/test-code/steps/4`

Environment variables are exposed to the override scripts.

Overrides are subject to the same timeouts as the original workflow entry.

Example:

1. Override all calls to actions/checkout

   name: Custom checkout
   
   overrides:
     jobs:
       "*":
         steps:
           uses: "^actions/checkout@"
           run: scripts/myCustomCheckoutCode.sh
   

Issue is stale and will be closed in 14 days unless there is new activity

Go away stale bot :P

To be honest, I read this like 5 times since it was opened and I still have no idea what exactly is proposed here.

Sounds like you want YAML linter/formatter which validates schema but also act running in dryrun mode

lol. What I tried to do was put the user story in the original post. Afterwards I followed up with a possible implementation spec, an RFC if you will.

The ultimate goal would be to have a way to use act as a sort of unit test framework for Actions Workflows. As I've dug into using act I found that almost everything I would need to do that already exists. The env.ACT var helps a lot towards this as it allows me to bypass certain parts of my workflow.

But what doesn't exist is a way to intercept the calls to specific Actions in a given workflow and run my own code in its place - aka stub those Actions. Thus I could tell the system "when this step is hit, execute this script instead of the actual action" and be able to test all parts of my workflows.

To simplify this, I want to know that, if my application unit tests pass, the deployment step will execute and if the unit tests fail, the deployment step will be skipped. If the unit tests pass, but the pull request is missing the "deployment" label, the deployment step would also be skipped.

Having the ability to mock the unit test step with a specific result, and then assert whether or not the deploy step would execute, without actually executing the steps, would theoretically make it much quicker to validate the logic in the workflows.

Some workflows can get pretty complex, so being able to tell what the output of step 12 would be if step 2 had input X would be pretty valuable. Hope this helps clarify.

why not leverage yq command merge functionallity to merge 2 github action yamls together?

In this case act should support passing github action yaml file as stdin like below

yq eval-all '. as $item ireduce ({}; . *+ $item )' build.yml build_override.yml | act issue_comment -W -

There are some caveats in the way yaml merging is done since github action file primarily contain arrays and the above native merge appends conflicting entries. The perfect merge solution is something that i am still figuring out with yq.

Ref

• https://mikefarah.gitbook.io/yq/operators/reduce#merge-all-yaml-files-together
• https://stackoverflow.com/a/67036496/3316017

@kishaningithub I'm not at all sure how that pertains to this idea. Please elucidate.

As far as I can see merging two YAML action workflows does not provide a mechanism for automatic validation via unit testing with mocking. Think mocha and sinon.

That said I'm sure the technique will be useful at some point, so thank you.