github-actions-runner-operator icon indicating copy to clipboard operation
github-actions-runner-operator copied to clipboard

Externals not mounted in DIND from Runner

Open bioshazard opened this issue 2 years ago • 5 comments

I am running into a complication where the mount of /home/runner/externals yields an empty folder in my jobs.

runner@runner-pool2-pod-4cj8m:~$ ls -la externals/total 16
drwxr-xr-x 4 runner 121 4096 Aug 30 17:28 .
drwxr-xr-x 1 runner 121 4096 Sep  1 12:01 ..
drwxr-xr-x 6 runner 121 4096 Aug 30 17:28 node12
drwxr-xr-x 3 runner 121 4096 Aug 30 17:28 node12_alpine
runner@runner-pool2-pod-4cj8m:~$ docker run --rm -it -v /home/runner/externals:/_externals --entrypoint bash centos:7
[root@b2aec0c0ed67 /]# ls -la /_externals/
total 8
drwxr-xr-x 2 root root 4096 Sep  1 12:25 .
drwxr-xr-x 1 root root 4096 Sep  1 12:25 ..

I have traced the issue down to DIND taking in that mount from its own container rather than mounting remotely from the runner as is apparently the expected function. I believe one could share a mount between runner and docker container to make it available on the DIND side of things, but out of the box, I do not see how any users are able to perform actions/checkout@v2 due to the missing /__e/node12/bin/node as the mount is empty.

Is anyone else's experience that the runner container can remotely mount its externals folder to DIND? Or is there some of other sync activity I am missing? Currently searching around for a clever way to expose the existing runner folder onto DIND, but I am starting to think I will rather need to copy the node12 folder to DIND to make it available when the mount is performed.

bioshazard avatar Sep 01 '21 12:09 bioshazard

hmmmm gonna try an init container to copy the externals to an emptydir mount...

bioshazard avatar Sep 01 '21 12:09 bioshazard

So using an init container totally worked. I created an emptyDir, mounted to an init container as /_externals, rsync'd /home/runner/externals, then mounted that volume in the runner and DIND and its now available. I am still not sure how anyone else is using this successfully given the above limitation. Am I missing something?

bioshazard avatar Sep 01 '21 13:09 bioshazard

I believe I am hitting this same issue on checkout.

OCI runtime exec failed: exec failed: container_linux.go:349: starting container process caused "exec: \"/__e/node12/bin/node\": stat /__e/node12/bin/node: no such file or directory": unknown

developerdino avatar Oct 05 '21 05:10 developerdino

@developerdino yep that is the exact error. You can see if you hop into the container that the externals folder has the node bin on the runner, but that DIND attempts the mount on its side instead mounting an empty folder.

bioshazard avatar Oct 05 '21 14:10 bioshazard

Thanks @bioshazard for all the hints as to how to fix this. I've managed to get everything working again based on your info by just creating the mounts for the externals, I'll create a PR for the example file so anyone else coming across this will be able to solve it easily.

I'm also wondering if this is a recent addition to the actions-runner deployment by Github and that is why we haven't seen this until only recently!

developerdino avatar Oct 05 '21 23:10 developerdino