Running `cirlceci local execute` inside a container fails

[fsproru]

• [x] I have read Contribution Guidelines.
• [x] I have checked for similar issues and haven't found anything relevant.
• [ ] This is not a security issue (which should be reported here: https://circleci.com/security/)

Do you want to request a feature or report a bug? Report a bug

What is the current behavior? Command errors out with the following:

Error: failed to start event processor: failed to compute task config: failed to read config file: read /tmp/local_build_config.yml: is a directory

Can you provide an example?

$ docker run -it --rm -v $(pwd):/data -v /var/run/docker.sock:/var/run/docker.sock --workdir /data alpine:3.8 ash
# apk add curl bash
# curl -fLSs https://circle.ci/cli | VERSION=0.1.7411 bash
# apk add --no-cache --upgrade git openssh ca-certificates docker
# circleci local execute
Docker image digest: sha256:29bd325e637508dec6c68d96fbcef2ddd2675eb98cf04805ec3fc39afa2c1ef4
Error: failed to start event processor: failed to compute task config: failed to read config file: read /tmp/local_build_config.yml: is a directory 

What is the expected behavior? The build job runs

Which version of the CLI and OS are you using? Did this work in previous versions? circleci-cli version 0.1.7411 os: macOS Catalina

Please provide the output of circleci version and circleci diagnostic.

circleci version
0.1.7411+93b9d40 (release)

circleci diagnostic

---
CircleCI CLI Diagnostics
---
Debugger mode: false
Config found: /root/.circleci/cli.yml
API host: https://circleci.com
API endpoint: graphql-unstable
Error: please set a token with 'circleci setup'
You can create a new personal API token here:
https://circleci.com/account/api

If you have any questions, feel free to ping us at @CircleCI-Public/x-team.

[fsproru]

Closed by mistake, reopening. That being said, this use case may not be supported per: https://github.com/CircleCI-Public/circleci-cli/pull/403

[bpg]

Similar issues: #381, #382

[pkolaczk]

Ubuntu 20.04, same issue.

$ circleci local execute  --job build 
Downloading latest CircleCI build agent...
Docker image digest: sha256:847ae88f32aaf7cfc04321b3eb25a7519630a949cf35df0d0b77ea1d7e886bd3
Error: failed to start event processor: failed to compute task config: failed to read config file: read /tmp/local_build_config.yml: is a directory

$ circleci version
0.1.7971+423e103 (release)
$ circleci diagnostic

---
CircleCI CLI Diagnostics
---
Debugger mode: false
Config found: /home/pkolaczk/.circleci/cli.yml
API host: https://circleci.com
API endpoint: graphql-unstable
OK, got a token.
Trying an introspection query on API... 
Ok.

[Aamirpal]

Hello guys, I am not sure why but when I removed the docker installed through snap(sudo snap remove docker) and re-install docker using this guide https://docs.docker.com/engine/install/ubuntu/

It WORKS for me!

I will find the reason and post here, till then try this solution it may work for you as well :smiley:

[andresespinosapc]

If I remove docker from snap, circleci commmand throws failed to connect to docker. How did you make it work?

[evictor]

The relevant file/line for this bug "failed to read config file: read /tmp/local_build_config.yml: is a directory" is local/local.go:295.

The problem is that the config file on the host that the CLI is supposed to create (but doesn't) is non-existent when the docker run command is called that tries to mount it as a volume. When some versions of Docker try to volume mount non-existent host file paths, they just assume you intended for the source mount to be a directory, so it is mounted as an empty directory in the container.

Note that I say "some versions of Docker." The reason why switching installation methods worked for you is probably because you inadvertently switched to a version of Docker that is defaulting to mounting that path as a file, which just so happens to work. Obviously this is not an acceptable solution.

I've never written a line of Go, so hopefully some CircleCI team member can come in here and fix this problem as I believe it will be a very easy fix and probably has to do with an async file write op that needs to be synchronous, or at least complete, before the subsequent docker run is called—so that the host config file exists and is correctly/reliably mounted as a file regardless of Docker version.

I will try to find time to submit a fix if I am able to spin up this env and test the bug effectively.

In the meantime, I came up with a real workaround for anyone else searching around on the Internet in which you recreate the Docker run command, except with a host config file that exists. ;)

# Replace values here as needed
cwd=$(pwd)
src_cfg="$cwd"/.circleci/config.yml
intermediate_cfg="$cwd"/.circleci/config_processed.yml # Have your VCS ignore this -- don't commit it
user_cfg="$HOME"/.circleci
execute_args="--job my-build-job" # Whatever args you intended to pass to execute
docker_sock=/var/run/docker.sock

# Note that in this command the volume mount source for
# /tmp/local_build_config.yml is absolute, and should exist
circleci config process $src_cfg > $intermediate_cfg \
  && docker run -it --rm -v $docker_sock:$docker_sock \
      -v $intermediate_cfg:/tmp/local_build_config.yml \
      -v "$cwd:$cwd" \
      -v "$user_cfg":/root/.circleci \
      --workdir "$cwd" \
      circleci/picard@sha256:ecbf254faa4688a254f9da7d2b576e46094c09f77d478cf091ca49cf587e8354 \
      circleci build --config /tmp/local_build_config.yml \
      $execute_args

[ECU-davidBradley]

I haven't been able to resolve the problem with the suggestion above. I was wondering if the real problem is at local.go:284. There is a file created (and I have verified it exists in /tmp) but it is never closed. There are two possible consequences of this:

1. depending on how GO works, the file might not exist at all after the call to "WriteString" (and even if it exists its contents are probably not flushed and so not usable by another process.
2. Depending on the permissions on the call to open the file in circleci, if it is locked for writing it may not be possible to open the file.

[danielloader]

Just ran into this problem when executing the CircleCI CLI inside a Visual Studio Code devcontainer - with docker from the host available inside the container. Just in case anyone stumbles on this from google, this is seemingly by design from reading the above.

Edit:

If you have this issue just change from docker-from-docker to docker-in-docker and it works fine with nested docker daemons/images in the parent container.

https://github.com/microsoft/vscode-dev-containers/issues/998

[williammoran]

I just ran into this issue walking through the basic intro tutorial here: https://circleci.com/blog/local-pipeline-development/

Ubuntu 20

I'm not sure what I can contribute to the issue that hasn't already been posted.

I do have a question, though: This is a blocking issue that hasn't been addressed in ~2 years. Why?

[williammoran]

Just to add a bit: I've been unable to find a workaround. I've tried multiple suggestions made across the internet and they either do nothing to fix the issue or result in different failures.

[mnitchev]

Looks like this is an issue only if you use snap. If you're looking for a quick workaround (and you can afford to not use snap), use the install.sh script in this repo to install the binary. For docker just install it whichever way is appropriate for your system. I used the get.docker.com script.

This is still an problem though, as the docs specifically list the snap method as the preferred one for Linux, granted the install.sh is listed as an alternative. Anyone following that doc will run into this issue, as I did myself.

[Pomax]

I'm using the brew installed circleci and and hitting this on an intel cpu macbook, so this isn't just snap. I am using colima (i.e. using the docker installed with brew install docker) rather than docker desktop, for obvious reasons.

[eldemcan]

its 2023 and I am having this problem as well. I removed snap version and installed script version unfortunately it didn't work

[juanjecilla-eb]

I am experiencing similar problems using Colima when trying to run a simple pipeline.