command not found (in image node:12.6-buster-slim, as opposed to GH Actions)

[fwilhe]

Hi,

as documented here GitHub configures passwordless sudo, so I can have sudo in my shell scripts to perform individual commands as root.

Given this minimal workflow:

on: push
name: Test
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - run: sudo id

The result on Actions is uid=0(root) gid=0(root) groups=0(root)

Running this workflow in act yields this result:

$ act
[Test/build] 🚀  Start image=node:12.6-buster-slim
[Test/build]   🐳  docker run image=node:12.6-buster-slim entrypoint=["/usr/bin/tail" "-f" "/dev/null"] cmd=[]
[Test/build]   🐳  docker cp src=/Users/fwilhe/temp/test-actions/. dst=/github/workspace
[Test/build] ⭐  Run sudo id
| /github/workflow/0: line 2: sudo: command not found
[Test/build]   ❌  Failure - sudo id
Error: exit with `FAILURE`: 127

Not sure what the most elegant solution would be without re-building the node:12.6-buster-slim image to install sudo and configure it passwordless. But I think the goal should be to be 1:1 compatible with actions, if possible.

[cplee]

See https://github.com/nektos/act#runners.

You can customize the image used for the runs-on. I've created an image that is a match to GitHub runners, but its like 15GB!

[fwilhe]

Hi @cplee

thanks for the work you put into this.

I know that the image is configurable, and I've locally tested with an image where I installed sudo. So this is kind of a workaround, but the reason I opened this issue is that this limits compatibility with workflows that run on GitHub. My intention was to point out where the documented (and actual) behavior of Actions is unequal the default of act, which makes it less useful compared to what it could be.

As mentioned I'm not sure what solution I would propose, as maintaining separate images seems inappropriate.

[cplee]

Tough problem. The runners that GitHub Actions offers are huge. I didn't feel right using an 18GB image as the default, requiring users of act to download that large of an image to run it.

I chose the node image because it was required to run node actions and still light enough. Not sure how to proceed...not sure what good enough looks like for the default docker image.

Open to ideas 🙏

[aidansteele]

I wonder if there's a way we could make an intermediary image that has 90% of the needed CLI tools for 10% of the footprint of the giant image. The only way we could reasonably do that (beyond intuition) is scanning a corpus of public GH Actions and seeing what works.

Then again, maybe it would give people a false sense of security as these issues would crop up less often - so it would be "stranger" and more likely to result in bug reports. 🤔

[aniongithub]

Maybe act could use a <Github-runner-name>.Dockerfile that the user can customize to make sure whatever tools they need is available in the used image. This could live in ~/.act/ or the current project directory?

[cplee]

see #196 for discussion on creating a slim runner

[christopherhesse]

I think at least the documentation should be updated, it currently states "The environment variables and filesystem are all configured to match what GitHub provides." but should say that that is only the case if you use this non-default image.

I think that docker for mac/windows is actually not very good with large images, but that an 18GB image in general isn't that crazy given the high speed internet and large disks available today.

[mobileka]

I would at least add common tools that are used in CI builds. For example, make is currently missing. I think that adding it doesn't change the size of the image that much but saves time and makes it much more convenient for a big crowd of people.

Or maybe I'm doing something wrong without realizing it and make is actually in the image :)

[fwilhe]

I would at least add common tools that are used in CI builds

I guess there is not enough agreement on what is "common". In the end you would include everything that github installs in the runner, which @cplee already did in that 20 gig image.

I have to say that building my own "runtime" image worked well for me.

The nice thing with that image is that you can also use it to run the whole action in via

jobs:
  build:
    runs-on: ubuntu-latest
    container: myimage

and to not rely on any "setup" actions. That makes it much easier to reproduce the github build on your local machine using act.

Hope that helps.

[mobileka]

I guess there is not enough agreement on what is "common"

Yes, I agree. But the initial argument against using the huge github image seems to be its size. Such tools as make are tiny and are incredibly popular in this context, so it's not clear to me what's the practical motivation of not including it into the default image.

That makes it much easier to reproduce the github build on your local machine using act.

In my opinion, most of the people interested in this project are interested in it because they already use github actions in their projects, not the other way around. Asking them to change their existing CI configuration (especially on a company/organization level) just to be able to test it with a particular tool doesn't seem to be practical.

Just my opinion though.

[fwilhe]

so it's not clear to me what's the practical motivation of not including it into the default image.

I'm not the author of act, so I can only guess that the motivation is to use an off-the-shelf image, and not to build one and then discuss with everyone what should be included or not.

Asking them to change their existing CI configuration (especially on a company level) just to be able to test it with a particular tool doesn't seem to be practical.

No, I'm not asking anyone to do that, I just say what worked for me.

[github-actions[bot]]

Issue is stale and will be closed in 7 days unless there is new activity

[sjiekak]

seems like git is not inside the node:12.6-buster-slim image either. Which is counter intuitive.

[gvolpe]

Got the same issue on NixOS, running this simple workflow.

[Build/Build] 🧪  Matrix: map[java:jdk11]
[Build/Build] 🚀  Start image=node:12.6-buster-slim
[Build/Build]   🐳  docker run image=node:12.6-buster-slim entrypoint=["/usr/bin/tail" "-f" "/dev/null"] cmd=[]
[Build/Build]   🐳  docker cp src=/home/gvolpe/workspace/scala3-typed-holes/. dst=/github/workspace
[Build/Build] ⭐  Run actions/[email protected]
[Build/Build]   ✅  Success - actions/[email protected]
[Build/Build] ⭐  Run Install Nix ❄️
[Build/Build]   ☁  git clone 'https://github.com/cachix/install-nix-action' # ref=v12
[Build/Build]   🐳  docker cp src=/home/gvolpe/.cache/act/cachix-install-nix-action@v12 dst=/actions/
| /actions/cachix-install-nix-action@v12/lib/install-nix.sh: line 11: sudo: command not found
| child_process.js:644
|     throw err;
|     ^
|
| Error: Command failed: /actions/cachix-install-nix-action@v12/lib/install-nix.sh
|     at checkExecSyncError (child_process.js:623:11)
|     at Object.execFileSync (child_process.js:641:15)
|     at Object.<anonymous> (/actions/cachix-install-nix-action@v12/lib/main.js:4:17)
|     at Module._compile (internal/modules/cjs/loader.js:776:30)
|     at Object.Module._extensions..js (internal/modules/cjs/loader.js:787:10)
|     at Module.load (internal/modules/cjs/loader.js:643:32)
|     at Function.Module._load (internal/modules/cjs/loader.js:556:12)
|     at Function.Module.runMain (internal/modules/cjs/loader.js:839:10)
|     at internal/main/run_main_module.js:17:11 {
|   status: 127,
|   signal: null,
|   output: [ null, null, null ],
|   pid: 28,
|   stdout: null,
|   stderr: null
| }
[Build/Build]   ❌  Failure - Install Nix ❄️
Error: exit with `FAILURE`: 1

[tyeth]

I propose we offer the large or slim image upon initial install (I used choco) and also include a configuration command (act --default-image slim/github/other) with the addition of --global affecting all images not the current workspace. Or something along those lines.

I'm just here to avoid never using act again because like mentioned above the readme clearly states it just works out the box and matches github actions like for like but locally in docker. This is not the case, and as a lazy user (I'd imagine 90% of your userbase) I would be happy to be asked once during install or just have to set my preferred default once and forget it. But to be honest I'm confused. In my head if you take the same docker image as github does and the action specify then yes it would be large but that's what the user signed up for. Still I feel your pain and appreciate the idea of slim by default. Also I dislike installers that don't have command line options like -y to answer yes to everything.

Separately, nice to see a healthy community and thanks for all the hard work / input all. Can't wait to see this move forward, but I'd be keen to hear if anyone had similar utility by using the official local github build agent, I imagine that's probably accessible/hackable enough (able to get access to terminal and files).

[catthehacker]

I'd be keen to hear if anyone had similar utility by using the official local github build agent, I imagine that's probably accessible/hackable enough (able to get access to terminal and files).

Don't use GitHub actions or run full GitHub Actions docker image then.

[tyeth]

How's about we enhance the readme to mention how to run the full image in the installation instructions. That will remove most of these "duplicate" issues. Basically it needs to be more than obvious that 99% of users will need to do this or face errors...

Having the sections after detailed usage instructions is a case of TLDR, and I did RTFM (mostly initially then fully after experiencing issues) but still missed it and then got my syntax wrong for my already instantiated act instance once I'd finally spotted it. The configuration file eventually solved my snafu 🤦

[catthehacker]

@tyeth Would something like this be okay?

[gvolpe]

I think that's fair @CatTheHacker . If possible, I would like to see another Small image with essential Unix tools such as sudo, curl, awk, etc.

[tyeth]

@CatTheHacker Brilliant yes, and maybe worth mentioning in that comment its using the Github official image for big (somehow more concisely). Thanks for taking a look at windows Path slash direction too.

[fwilhe]

It might make sense to also base the "small" image on Ubuntu. I noticed that the setup-python action for example relies on ubuntu specifically to find out which archive to download, and fails on the node image that is based on debian.

[catthehacker]

I think that's fair @CatTheHacker . If possible, I would like to see another Small image with essential Unix tools such as sudo, curl, awk, etc.

@gvolpe that's not in my power, to me it also seems weird that image without sudo or curl was chosen as default. I can make such as default in my fork but it will be up to act owner if they will chose to implement that in the end.

@CatTheHacker Brilliant yes, and maybe worth mentioning in that comment its using the Github official image for big (somehow more concisely). Thanks for taking a look at windows Path slash direction too.

@tyeth There isn't any official GitHub image for Docker unless you mean act big one. I'm open for suggestions on how to write/convey the message 👍 since I'm usually not the best in describing things to people.

It might make sense to also base the "small" image on Ubuntu. I noticed that the setup-python action for example relies on ubuntu specifically to find out which archive to download, and fails on the node image that is based on debian.

@fwilhe Just, Yes. (even though I'm always for Debian and ~~hate~~ strongly dislike Ubuntu, but when using ubuntu-xx.xx runner, I expect it to be actually Ubuntu)

feel free to make any suggestions, I'm trying to do as much as possible for this repo in free time

[tyeth]

@tyeth Would something like this be okay?

My english brain is not firing on all cylinders right now, but having seen the merge progress today I thought I'd better get back here sharpish 👀 Maybe just add to the brackets describing big, from (includes more tools) to (included tools match Github Actions)

But I'd feel better with something else 🤦 Any suggestions people?

[catthehacker]

Ok, so I've been thinking a bit and came up with this

• Micro image: current node:12.6-buster-slim (we could build upon that image to add below image env vars but leave it with just node.js, to make it a bit more compatible)
• Medium image: https://github.com/CatTheHacker/docker-images, ubuntu:act-*, includes required env vars for some software setup (like ruby/setup-ruby), includes more tools (gawk,curl,git,ssh,etc.)
• Large image: nektos/act-environments-ubuntu:18.04
• I also have been working on full GitHub Actions Runner image (+60GB), and I think of making above large image as big and mine as large (quay.io/catthehacker/virtual-environments | https://github.com/CatTheHacker/virtual-environments)

Looking for feedback/suggestions

[xloem]

It would be really nice if there were clear instructions on how a user can build and enable an image of their own that matches the one used by github. Here's are two clear question to answer:

• How do I configure act to build my workflows without modification?
• How do I configure act to use a docker image I build myself? I'm not very familiar with docker.

[catthehacker]

@xloem

• What modification?
• https://github.com/nektos/act#use-an-alternative-runner-image

[xloem]

Modifications to remove or install inaccessible commands. I'll ask the first question a different way:

• How do I build a complete docker github workflow image?

I infer the way to run a custom docker image is:

• Make an empty folder
• Make a Dockerfile in the folder based on a simple template (needs link to example)
• Add lines to the Dockerfile to install any missing packages (needs example format)
• Build the Docker file into an image (needs commandline)
• Change your .actrc to reference the name of the image like nektos/act-environments-ubuntu:18.04 in -P ubuntu-18.04=nektos/act-environments-ubuntu:18.04

[catthehacker]

How do I build a complete docker github workflow image?

It's just not possible due to the way how Docker works. See https://github.com/actions/virtual-environments/discussions/2320#discussioncomment-328445

way to run a custom docker image

I'd say it's not our responsibility to teach how to make Docker images, there is plenty of (much better) sources that go into details on how to do that where official Docker documentation is quite good source on that. README.md describes:

• It requires Docker
• How to run different Docker image

Eventually there could be link to Docker documentation regarding how to make Docker images in README.md.

[killthekitten]

I was debugging another issue and wanted to check if the large image would solve it. It looks like I'm stuck with the micro image that I've chosen at the very first install. Is there a way to force-reset to the large one?

I've tried a bunch of things:

• Reinstalling act
• Deleting ~/.actrc
• Deleting ~/.cache/act
• Providing -P ... with the command
• Deleting volumes and containers

After reinstalling it fails almost instantly and never attempts to download the large image, and it looks like containers section of the workflow overrides the runs-on.

Logs

DEBU[0000] Loading environment from /Users/user/project/.env 
DEBU[0000] Loading secrets from /Users/user/project/.act.secrets 
DEBU[0000] Loading workflows from '/Users/user/project/.github/workflows' 
DEBU[0000] Reading workflow '/Users/user/project/.github/workflows/ci.yml' 
DEBU[0000] Reading workflow '/Users/user/project/.github/workflows/deploy-production.yaml' 
DEBU[0000] Reading workflow '/Users/user/project/.github/workflows/deploy-staging.yaml' 
DEBU[0000] Planning job: test                           
DEBU[0000] Loading slug from git directory '/Users/user/project/.git' 
DEBU[0000] Found revision: efaea79abd1368aab365933c69b326d3542a9bd5 
DEBU[0000] Loading revision from git directory '/Users/user/project/.git' 
DEBU[0000] Found revision: efaea79abd1368aab365933c69b326d3542a9bd5 
DEBU[0000] HEAD points to 'efaea79abd1368aab365933c69b326d3542a9bd5' 
DEBU[0000] HEAD matches refs/heads/main                 
DEBU[0000] using github ref: refs/heads/main            
DEBU[0000] context env => map[ACT:true] 
[Python application/test] 🚀  Start image=python:3.8.5
DEBU[0000] Loading slug from git directory '/Users/user/project/.git' 
DEBU[0000] Found revision: efaea79abd1368aab365933c69b326d3542a9bd5 
DEBU[0000] Loading revision from git directory '/Users/user/project/.git' 
DEBU[0000] Found revision: efaea79abd1368aab365933c69b326d3542a9bd5 
DEBU[0000] HEAD points to 'efaea79abd1368aab365933c69b326d3542a9bd5' 
DEBU[0000] HEAD matches refs/heads/main                 
DEBU[0000] using github ref: refs/heads/main            
[Python application/test]   🐳  docker pull python:3.8.5
DEBU[0000] Image exists? true                           
[Python application/test]   🐳  docker volume rm act-Python-application-test
[Python application/test]   🐳  docker create image=python:3.8.5 platform=linux/amd64 entrypoint=["/usr/bin/tail" "-f" "/dev/null"] cmd=[]
[Python application/test] Created container name=act-Python-application-test id=47772b3cbce77238f1debd13522cb1457c0bbc2727de148e9a80577d88f4d837 from image python:3.8.5 (platform: linux/amd64)
[Python application/test] ENV ==> [RUNNER_TOOL_CACHE=/opt/hostedtoolcache RUNNER_OS=Linux RUNNER_TEMP=/tmp]
[Python application/test]   🐳  docker run image=python:3.8.5 platform=linux/amd64 entrypoint=["/usr/bin/tail" "-f" "/dev/null"] cmd=[]
[Python application/test] Starting container: 47772b3cbce77238f1debd13522cb1457c0bbc2727de148e9a80577d88f4d837
[Python application/test] Started container: 47772b3cbce77238f1debd13522cb1457c0bbc2727de148e9a80577d88f4d837
[Python application/test]   🐳  docker cp src=/Users/user/project/. dst=/Users/user/project
[Python application/test] Exec command '[mkdir -p /Users/user/project]'
DEBU[0000] Writing tarball /var/folders/5b/k1by84y17ln1mxfr8nczl3hw0000gn/T/act438170973 from /Users/user/project/. 
DEBU[0000] Stripping prefix:/Users/user/project/ src:/Users/user/project/. 
[Python application/test] Extracting content from '/var/folders/5b/k1by84y17ln1mxfr8nczl3hw0000gn/T/act438170973' to '/Users/user/project'
DEBU[0006] Writing entry to tarball workflow/event.json len:2 
DEBU[0006] Writing entry to tarball workflow/envs.txt len:0 
DEBU[0006] Writing entry to tarball home/.act len:0     
[Python application/test] Extracting content to '/Users/user/project'
DEBU[0006] Loading slug from git directory '/Users/user/project/.git' 
DEBU[0006] Found revision: efaea79abd1368aab365933c69b326d3542a9bd5 
DEBU[0006] Loading revision from git directory '/Users/user/project/.git' 
DEBU[0006] Found revision: efaea79abd1368aab365933c69b326d3542a9bd5 
DEBU[0006] HEAD points to 'efaea79abd1368aab365933c69b326d3542a9bd5' 
DEBU[0006] HEAD matches refs/heads/main                 
DEBU[0006] using github ref: refs/heads/main            
DEBU[0006] Loading slug from git directory '/Users/user/project/.git' 
DEBU[0006] Found revision: efaea79abd1368aab365933c69b326d3542a9bd5 
DEBU[0006] Loading revision from git directory '/Users/user/project/.git' 
DEBU[0006] Found revision: efaea79abd1368aab365933c69b326d3542a9bd5 
DEBU[0006] HEAD points to 'efaea79abd1368aab365933c69b326d3542a9bd5' 
DEBU[0006] HEAD matches refs/heads/main                 
DEBU[0006] using github ref: refs/heads/main            
DEBU[0006] context env => map[ACT:true] 
DEBU[0006] context env => map[ACT:true] 
[Python application/test] setupEnv => map[ACT:true]
[Python application/test] ⭐  Run actions/checkout@v2
DEBU[0006] Loading slug from git directory '/Users/user/project/.git' 
DEBU[0006] Found revision: efaea79abd1368aab365933c69b326d3542a9bd5 
DEBU[0006] Loading revision from git directory '/Users/user/project/.git' 
DEBU[0006] Found revision: efaea79abd1368aab365933c69b326d3542a9bd5 
DEBU[0006] HEAD points to 'efaea79abd1368aab365933c69b326d3542a9bd5' 
DEBU[0006] HEAD matches refs/heads/main                 
DEBU[0006] using github ref: refs/heads/main            
[Python application/test] Skipping actions/checkout
[Python application/test]   ✅  Success - actions/checkout@v2
DEBU[0006] Loading slug from git directory '/Users/user/project/.git' 
DEBU[0006] Found revision: efaea79abd1368aab365933c69b326d3542a9bd5 
DEBU[0006] Loading revision from git directory '/Users/user/project/.git' 
DEBU[0006] Found revision: efaea79abd1368aab365933c69b326d3542a9bd5 
DEBU[0006] HEAD points to 'efaea79abd1368aab365933c69b326d3542a9bd5' 
DEBU[0006] HEAD matches refs/heads/main                 
DEBU[0006] using github ref: refs/heads/main            
DEBU[0006] Loading slug from git directory '/Users/user/project/.git' 
DEBU[0007] Found revision: efaea79abd1368aab365933c69b326d3542a9bd5 
DEBU[0007] Loading revision from git directory '/Users/user/project/.git' 
DEBU[0007] Found revision: efaea79abd1368aab365933c69b326d3542a9bd5 
DEBU[0007] HEAD points to 'efaea79abd1368aab365933c69b326d3542a9bd5' 
DEBU[0007] HEAD matches refs/heads/main                 
DEBU[0007] using github ref: refs/heads/main            
DEBU[0007] context env => map[ACT:true] 
DEBU[0007] Loading slug from git directory '/Users/user/project/.git' 
DEBU[0007] Found revision: efaea79abd1368aab365933c69b326d3542a9bd5 
DEBU[0007] Loading revision from git directory '/Users/user/project/.git' 
DEBU[0007] Found revision: efaea79abd1368aab365933c69b326d3542a9bd5 
DEBU[0007] HEAD points to 'efaea79abd1368aab365933c69b326d3542a9bd5' 
DEBU[0007] HEAD matches refs/heads/main                 
DEBU[0007] using github ref: refs/heads/main            
DEBU[0007] context env => map[ACT:true] 
DEBU[0007] Evaluating 'secrets['SOME_SECRET_KEY']' instead of 'secrets.SOME_SECRET_KEY' 
DEBU[0007] context env => map[ACT:true] 
DEBU[0007] Evaluating 'secrets['SOME_SECRET_KEY']' instead of 'secrets.SOME_SECRET_KEY' 
[Python application/test] setupEnv => map[ACT:true]
[Python application/test] ⭐  Run webfactory/[email protected]
[Python application/test]   ☁  git clone 'https://github.com/webfactory/ssh-agent' # ref=v0.5.2
[Python application/test]   cloning https://github.com/webfactory/ssh-agent to /Users/user/.cache/act/[email protected]
[Python application/test] Cloned https://github.com/webfactory/ssh-agent to /Users/user/.cache/act/[email protected]
[Python application/test] Checked out v0.5.2
DEBU[0008] Read action &{webfactory/ssh-agent  Run `ssh-agent` and load an SSH key to access other private repositories map[ssh-auth-sock:{Where to place the SSH Agent auth socket false } ssh-private-key:{Private SSH key to register in the SSH agent true }] map[] {node12 map[] dist/index.js  [] []} {yellow loader}} from '/Users/user/.cache/act/[email protected]/action.yml' 
DEBU[0008] About to run action &{webfactory/ssh-agent  Run `ssh-agent` and load an SSH key to access other private repositories map[ssh-auth-sock:{Where to place the SSH Agent auth socket false } ssh-private-key:{Private SSH key to register in the SSH agent true }] map[] {node12 map[] dist/index.js  [] []} {yellow loader}} 
DEBU[0008] type=3 actionDir=/Users/user/.cache/act/[email protected] actionPath= Workdir=/Users/user/project ActionCacheDir=/Users/user/.cache/act [email protected] containerActionDir=/actions 
DEBU[0008] Removing /Users/user/.cache/act/[email protected]/.gitignore before docker cp 
[Python application/test]   🐳  docker cp src=/Users/user/.cache/act/[email protected] dst=/actions/
[Python application/test] Exec command '[mkdir -p /actions/]'
DEBU[0008] Writing tarball /var/folders/5b/k1by84y17ln1mxfr8nczl3hw0000gn/T/act810834712 from /Users/user/.cache/act/[email protected] 
DEBU[0008] Stripping prefix:/Users/user/.cache/act/ src:/Users/user/.cache/act/[email protected] 
[Python application/test] Extracting content from '/var/folders/5b/k1by84y17ln1mxfr8nczl3hw0000gn/T/act810834712' to '/actions/'
DEBU[0008] executing remote job container: [node /actions/[email protected]/dist/index.js] 
[Python application/test] Exec command '[node /actions/[email protected]/dist/index.js]'
| OCI runtime exec failed: exec failed: container_linux.go:367: starting container process caused: exec: "node": executable file not found in $PATH: unknown
[Python application/test]   ❌  Failure - webfactory/[email protected]
DEBU[0008] exit with `FAILURE`: 126                     
DEBU[0008] exit with `FAILURE`: 126                     
DEBU[0008] exit with `FAILURE`: 126                     
Error: exit with `FAILURE`: 126

[catthehacker]

@killthekitten You are using python:3.8.5 according to your log. To get default image survey, .actrc cannot exist in ~, ~/.config/ and current directory where act is run or you can just replace image yourself, those are described in README