act
act copied to clipboard
nektos
feat: DRYRUN job output comments
Ideas are welcome, this is already working
Run with --dryrun, it throws before this change
on: push
jobs:
list-manifests:
runs-on: ubuntu-latest
outputs:
# Comment for this output
# DRYRUN: ["7.3", "7.4"]
# Another Comment for this output
matrix: ${{ steps.set-matrix.outputs.matrix }}
steps:
- run: echo Hi
otherjob:
needs: list-manifests
runs-on: ubuntu-latest
strategy:
matrix:
X: ${{ fromjson(needs.list-manifests.outputs.matrix) }}
steps:
- run: echo Hi
- [ ] Discuss this feature
- [x] Add Test
Fixes #1347
Codecov Report
Merging #1350 (7a727d0) into master (4f8da0a) will increase coverage by
3.00%. The diff coverage is74.79%.
@@ Coverage Diff @@
## master #1350 +/- ##
==========================================
+ Coverage 57.50% 60.51% +3.00%
==========================================
Files 32 44 +12
Lines 4594 7035 +2441
==========================================
+ Hits 2642 4257 +1615
- Misses 1729 2469 +740
- Partials 223 309 +86
| Impacted Files | Coverage Δ | |
|---|---|---|
| pkg/model/action.go | 0.00% <0.00%> (ø) |
|
| pkg/model/step_result.go | 0.00% <ø> (ø) |
|
| pkg/container/docker_run.go | 12.82% <26.66%> (+7.27%) |
:arrow_up: |
| pkg/container/docker_pull.go | 33.33% <33.33%> (ø) |
|
| pkg/container/file_collector.go | 37.30% <35.45%> (ø) |
|
| pkg/common/git/git.go | 50.00% <46.80%> (ø) |
|
| pkg/model/workflow.go | 48.58% <47.16%> (-2.33%) |
:arrow_down: |
| pkg/container/docker_auth.go | 47.61% <50.00%> (+2.61%) |
:arrow_up: |
| pkg/exprparser/interpreter.go | 73.37% <53.48%> (-0.02%) |
:arrow_down: |
| pkg/runner/runner.go | 76.47% <64.44%> (ø) |
|
| ... and 39 more |
:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more
![codecov[bot] avatar](https://avatars.githubusercontent.com/in/254?v=4 "Published by a pub.dev verified publisher"){kind=small}
🦙 MegaLinter status: ✅ SUCCESS
| Descriptor | Linter | Files | Fixed | Errors | Elapsed time |
|---|---|---|---|---|---|
| ✅ EDITORCONFIG | editorconfig-checker | 3 | 0 | 0.02s | |
| ✅ REPOSITORY | gitleaks | yes | no | 2.38s | |
| ✅ REPOSITORY | git_diff | yes | no | 0.0s | |
| ✅ REPOSITORY | secretlint | yes | no | 0.97s |
See errors details in artifact MegaLinter reports on CI Job page
Set VALIDATE_ALL_CODEBASE: true in mega-linter.yml to validate all sources, not only the diff
MegaLinter is graciously provided by 
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
I may be using this wrong but I got the following:
act -n
panic: runtime error: slice bounds out of range [:-1] [recovered]
panic: runtime error: slice bounds out of range [:-1]
goroutine 1 [running]:
gopkg.in/yaml%2ev3.handleErr(0xc0002df830)
/Users/nwaters/go/pkg/mod/gopkg.in/[email protected]/yaml.go:294 +0x6d
panic({0x17768e0, 0xc00003a210})
/usr/local/go/src/runtime/panic.go:884 +0x212
github.com/nektos/act/pkg/model.(*Job).UnmarshalYAML(0xc00014a400, 0x175db00?)
/Users/nwaters/GitHub/act/pkg/model/workflow.go:105 +0x425
gopkg.in/yaml%2ev3.(*decoder).callUnmarshaler(0xc000128310, 0x175db00?, {0x5c52898?, 0xc00014a400?})
/Users/nwaters/go/pkg/mod/gopkg.in/[email protected]/decode.go:366 +0x33
gopkg.in/yaml%2ev3.(*decoder).prepare(0xc0002dee80?, 0x10c8511?, {0x175db00?, 0xc000014170?, 0x1e7e1a0?})
/Users/nwaters/go/pkg/mod/gopkg.in/[email protected]/decode.go:423 +0x286
gopkg.in/yaml%2ev3.(*decoder).unmarshal(0xc000128310, 0xc000152960, {0x175db00?, 0xc000014170?, 0xc0001125d0?})
/Users/nwaters/go/pkg/mod/gopkg.in/[email protected]/decode.go:502 +0x2a5
gopkg.in/yaml%2ev3.(*decoder).mapping(0xc000128310, 0xc000152820, {0x16f04e0?, 0xc000134390?, 0xc000152280?})
/Users/nwaters/go/pkg/mod/gopkg.in/[email protected]/decode.go:848 +0xa07
gopkg.in/yaml%2ev3.(*decoder).unmarshal(0xc000128310, 0xc000152820, {0x16f04e0?, 0xc000134390?, 0xc00014a168?})
/Users/nwaters/go/pkg/mod/gopkg.in/[email protected]/decode.go:510 +0x3eb
gopkg.in/yaml%2ev3.(*decoder).mappingStruct(0xc000128310, 0xc000152000, {0x177b740?, 0xc0001342d0?, 0x1674131?})
/Users/nwaters/go/pkg/mod/gopkg.in/[email protected]/decode.go:935 +0xeec
gopkg.in/yaml%2ev3.(*decoder).mapping(0xc000128310, 0xc000152000, {0x177b740?, 0xc0001342d0?, 0x0?})
/Users/nwaters/go/pkg/mod/gopkg.in/[email protected]/decode.go:786 +0xbe
gopkg.in/yaml%2ev3.(*decoder).unmarshal(0xc000128310, 0xc000152000, {0x177b740?, 0xc0001342d0?, 0x0?})
/Users/nwaters/go/pkg/mod/gopkg.in/[email protected]/decode.go:510 +0x3eb
gopkg.in/yaml%2ev3.(*decoder).document(0x0?, 0x0?, {0x177b740?, 0xc0001342d0?, 0xc00014a000?})
/Users/nwaters/go/pkg/mod/gopkg.in/[email protected]/decode.go:527 +0x5b
gopkg.in/yaml%2ev3.(*decoder).unmarshal(0xc000128310, 0xc000001f40, {0x177b740?, 0xc0001342d0?, 0x0?})
/Users/nwaters/go/pkg/mod/gopkg.in/[email protected]/decode.go:498 +0x42d
gopkg.in/yaml%2ev3.(*Decoder).Decode(0xc0002df8a8, {0x170ca20?, 0xc0001342d0})
/Users/nwaters/go/pkg/mod/gopkg.in/[email protected]/yaml.go:131 +0x236
github.com/nektos/act/pkg/model.ReadWorkflow({0x18d7c00?, 0xc000014060})
/Users/nwaters/GitHub/act/pkg/model/workflow.go:497 +0x1ca
github.com/nektos/act/pkg/model.NewWorkflowPlanner({0xc00003e400?, 0x17c666c?}, 0x0)
/Users/nwaters/GitHub/act/pkg/model/planner.go:130 +0x58e
github.com/nektos/act/cmd.newRunCommand.func1(0xc000004280, {0xc0001282a0, 0x0, 0xc?})
/Users/nwaters/GitHub/act/cmd/root.go:302 +0x2a5
github.com/spf13/cobra.(*Command).execute(0xc000004280, {0xc000136000, 0x7, 0xc})
/Users/nwaters/go/pkg/mod/github.com/spf13/[email protected]/command.go:872 +0x694
github.com/spf13/cobra.(*Command).ExecuteC(0xc000004280)
/Users/nwaters/go/pkg/mod/github.com/spf13/[email protected]/command.go:990 +0x3bd
github.com/spf13/cobra.(*Command).Execute(...)
/Users/nwaters/go/pkg/mod/github.com/spf13/[email protected]/command.go:918
github.com/nektos/act/cmd.Execute({0x18dd180, 0xc0003c0980}, {0x18d61a0, 0x11})
/Users/nwaters/GitHub/act/cmd/root.go:85 +0xd9f
main.main()
/Users/nwaters/GitHub/act/main.go:34 +0x1a7
when using the following in the workflow:
jobs:
list-manifests:
runs-on: ubuntu-latest
outputs:
# DRYRUN: ["rgi,5.2.1", "redcap,1.0.0"]
matrix: ${{ steps.set-matrix.outputs.matrix }}
steps:
- uses: actions/checkout@v2
- id: set-matrix
run: echo "::set-output name=matrix::$(cat build_manifest.csv |grep -v '^#' | jq --raw-input . | jq -c --slurp .)"
build:
name: 'Build'
needs: list-manifests
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
strategy:
matrix:
manifest: ${{ fromJson(needs.list-manifests.outputs.matrix) }}
steps:
- name: "Build:checkout"
uses: actions/checkout@v2
- name: Get version
env:
tool_ver: ${{ matrix.manifest }}
id: versplit
run: echo "::set-output name=VERSION::${tool_ver##*,}"
Am I using the syntax you envisioned correctly?
The syntax you used should work if this PR would work correctly. To avoid the crash, please add a comment line above and under the DRYRUN:, until this is corrected.
#
# DRYRUN: ["rgi,5.2.1", "redcap,1.0.0"]
#
matrix: ${{ steps.set-matrix.outputs.matrix }}
Seems like I expected ("#DRYRUN")[0:-1] to work in go.
PR is stale and will be closed in 14 days unless there is new activity
@nektos/act-maintainers Do we want this kind of feature? If we want it, I would fix the remaining bugs and add Tests.
It would also be nice to allow using another yaml file to set the dryrun outputs of jobs, however then we somehow need to link the dryrun definition files with actual workflow files.
I'm undecided. I can see this has some value but it also will increase the maintenance overhead. Also the burden to explain the differences between act and github-actions might increase as people do not write an override file.
I would try to stick to the comment approach for now (if any).
I like the comment approach, simpler to keep everything in a single file. With the amount of complexity that act/github-actions can handle this makes development much easier for me.
I can't stress enough how amazing this feature would be to have. Github does a really really poor job of helping people run their workflows since you have to actually run them on Github with real events. Testing workflows is even more of a nightmare. If we can get this feature merged (with override file or not) it will enable me to test my 2k LOC workflow in seconds with so much ease of trying any scenario I want (just set different overrides and gh events).
The problem with my current comment approach is that you cannot write multiline outputs
it will enable me to test my 2k LOC workflow
I'm not shure if this really allows to actually test your complex workflow, because DRYRUN ( a barely maintained feature of act, I broke local actions in this mode and it was noticed 1 year after my change were merged ) doesn't test most parts of your workflow.
Also the burden to explain the differences between act and github-actions might increase
I even had the idea to remove DRYRUN, but I didn't attempt to propose that in a PR yet. The actions/runner doesn't even know what dryrun is. I honestly don't actually use DRYRUN myself, other than testing bug reports.
Same here. I never use dry run. Not sure what it even should do and what dry means in the context of act.
Actions can implement such a mode and just output what they are supposed to do. But for act this is strange.
Same here. I never use dry run. I honestly don't actually use DRYRUN myself, other than testing bug reports.
Testing is exactly why I need dryrun so I can test scenarios like bug reports (except, I'm testing that the bug doesn't exist).
Not sure what it even should do and what dry means in the context of act.
The current implementation just evaluates the workflow logic that Github has created and logs "I would be running step abc". This breaks if outputs are evaluated but steps aren't actually ran. By being able to mock the outputs like in this PR, that completes the dryrun story for me.
I'm not sure the architectural complexity this feature may add but it honestly is a big deal for me.
Actions can implement such a mode and just output what they are supposed to do.
This was a card I was hoping to not have to use tbh but understand how some may think this is appropriate.
@ChristopherHX, interested to get some clarification on "doesn't test most parts of your workflow."
interested to get some clarification on "doesn't test most parts of your workflow
- Step outputs and env are missing
- a problem if the step if expression results in an error if something is missing, like using fromjson
- skips entire local actions, as far I know not possible to fix if the origin of the local action is not in the your source repository
- Just my opinon dryrun was not my idea, it already existed in act.
I have a big stale PR, so I want to confirm if this design is compatible with the opinon of the owner cplee., I have no problem adding this to act.
I prefer to add the needs context as a cmdline option containing a yaml structure with the result of the skipped dependency. This allows to unit test a specfic test, while actually run all the steps.
For example:
system.act.skipDependencies: true
# For github actions, default for all jobids
system.act.needs: |
previous:
output:
out1: val2
result: failure
# Allow to correctly set dependent jobid
system.act.<grandparentjobid/parentjobid/jobid>.needs: |
previous:
output:
out1: val2
result: failure
Also the job result is read from the provided data, if the result is missing use 'success' with an incomplete needs ctx.
@ChristopherHX I've realized how I can achieve dryruns and controlled outputs without act supporting it. Since I'm making a tool to run workflows with act, I can just modify the workflow before calling act.
Given the following steps we want to test:
steps:
- name: Do thing
id: expensive-function
uses: actions/heavy-computer
- name: Get expensive computation
run: echo "The output is ${{ steps.expensive-function.outputs.COMPUTED }}"
my wrapper will modify this file like so:
steps:
- name: Do thing
id: expensive-function
run: echo "COMPUTED=abc123" >> $GITHUB_OUTPUT
- name: Get thing
run: echo "The output is ${{ steps.expensive-function.outputs.COMPUTED }}"
This might seem weird to want to test this but for more complex workflows which run actions based on outputs we want to make sure that something happens.
- name: Generate changelog
if: inputs.disable_versioning != true && steps.expensive-function.outputs.COMPUTED == 'abc123'
run: |
(cd /tmp
wget https://github.com/mikefarah/yq/releases/download/3.0.1/yq_linux_amd64 -O yq
echo "a1097c74b81a2ef255583d9718bf4be6 yq" | md5sum -c -
chmod +x yq
PATH="${PWD}:${PATH}" GH_TOKEN=${{ secrets.FLOWZONE_TOKEN }} $(npm root -g)/versionist/scripts/generate-changelog.sh "${GITHUB_WORKSPACE}"
)
Testing the actual run code can be achieved by having the workflow reference local scripts... for example, the above changelog step should be converted to:
- name: Generate changelog
if: inputs.disable_versioning != true && github.event_name == 'pull_request'
run: "${GITHUB_WORKSPACE}/.github/generate_changelog.sh"
Then, you write tests for that shell script separate from testing the entire Workflow (integration vs unit style).
I'm pinging you for closure on this feature since I won't pursue advocating it anymore given the lack of interest / complexity.
| /var/run/act/workflow/expensive-function: line 2: $GITHUB_OUTPUT: ambiguous redirect
aww..
edit: I saw https://github.com/nektos/act/issues/1386.
Anyway, I fixed the bugs of this change and added a test. It's now up to the other maintainer to decide.
I would use this feature if it's merged but I've been able to implement it by running a script that overrides every step's uses/run with a dummy run step (like just echoing).
I'm undecided since I never use this feature. Therefore I leave it to the others to decide.
Abandoned based on maintainer opinion
- neutral
- neutral
- no
I made a backup of this PR under my fork of act https://github.com/ChristopherHX/act/tree/dryrun-output-comments