docker
ERROR: failed to solve: process "/bin/sh
Contributing guidelines
- [X] I've read the contributing guidelines and wholeheartedly agree
I've found a bug and checked that ...
- [X] ... the documentation does not mention anything about my problem
- [X] ... there are no open or closed issues that are related to my problem
Description
Hi,
i am getting an error with buildx only. When running the same Dockerfile with docker build it works fine.
Expected behaviour
The image should build.
Actual behaviour
The image doesn't build.
Buildx version
github.com/docker/buildx v0.11.2 9872040b6626fb7d87ef7296fd5b832e8cc2ad17
Docker info
Client: Docker Engine - Community
Version: 24.0.5
Context: default
Debug Mode: false
Plugins:
compose: Docker Compose (Docker Inc.)
Version: 2.20.2
Path: /Users/groebroeck/.docker/cli-plugins/docker-compose
Server:
Containers: 21
Running: 1
Paused: 0
Stopped: 20
Images: 128
Server Version: 20.10.20
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 9cd3357b7fd7218e4aec3eae239db1f68a5a6ec6
runc version: 5fd4c4d144137e991c4acebb2146ab1483a97925
init version:
Security Options:
seccomp
Profile: default
Kernel Version: 5.15.82-0-virt
Operating System: Alpine Linux v3.16
OSType: linux
Architecture: aarch64
CPUs: 2
Total Memory: 1.93GiB
Name: colima
ID: 767Q:HZTI:3O2Y:ZSL5:PA2I:7ZE7:JE3N:RFPW:RIY6:7YAV:LRRU:XJH5
Docker Root Dir: /var/lib/docker
Debug Mode: false
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Builders list
NAME/NODE DRIVER/ENDPOINT STATUS BUILDKIT PLATFORMS
hopeful_wright * docker-container
hopeful_wright0 unix:///Users/groebroeck/.colima/docker.sock running v0.11.6 linux/arm64, linux/amd64, linux/amd64/v2
colima docker
colima colima running v0.8.2+c0149372 linux/arm64, linux/amd64
default docker
default default running v0.8.2+c0149372 linux/arm64, linux/amd64
Configuration
FROM eclipse-temurin:19-jre as builder
ARG JAR=komga-1.3.0.jar
COPY assembly/* /
RUN java -Djarmode=layertools -jar ${JAR} extract
FROM eclipse-temurin:19-jre
# Install libjxl on x64 only. Homebrew is not available on other architectures.
RUN if [ "`uname -m`" = "x86_64" ]; then \
apt -y update && \
apt -y install git gcc && \
NONINTERACTIVE=1 /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" && \
eval "$(/home/linuxbrew/.linuxbrew/bin/brew shellenv)" && \
brew install jpeg-xl; fi
VOLUME /tmp
VOLUME /config
WORKDIR app
COPY --from=builder dependencies/ ./
COPY --from=builder spring-boot-loader/ ./
COPY --from=builder snapshot-dependencies/ ./
COPY --from=builder application/ ./
ENV KOMGA_CONFIGDIR="/config"
ENV LC_ALL=en_US.UTF-8
ENV LD_LIBRARY_PATH="${LD_LIBRARY_PATH}:/home/linuxbrew/.linuxbrew/lib/"
ENTRYPOINT ["java", "--enable-preview", "--enable-native-access=ALL-UNNAMED", "org.springframework.boot.loader.JarLauncher", "--spring.config.additional-location=file:/config/"]
EXPOSE 25600
LABEL org.opencontainers.image.source="https://github.com/gotson/komga"
Build logs
docker-buildx build --platform linux/amd64,linux/arm/v7,linux/arm64/v8 --no-cache --file Dockerfile . Thu Aug 3 18:36:31 2023
[+] Building 2.0s (12/33) docker-container:hopeful_wright
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 1.18kB 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [linux/amd64 internal] load metadata for docker.io/library/eclipse-temurin:19-jre 1.7s
=> [linux/arm64 internal] load metadata for docker.io/library/eclipse-temurin:19-jre 1.7s
=> [linux/arm/v7 internal] load metadata for docker.io/library/eclipse-temurin:19-jre 1.7s
=> CACHED [linux/arm/v7 stage-1 1/7] FROM docker.io/library/eclipse-temurin:19-jre@sha256:e7934eaf29182fa4f883e4cc28a127fdd70e652f8208d3c8dc4edc3bfc92fb49 0.0s
=> => resolve docker.io/library/eclipse-temurin:19-jre@sha256:e7934eaf29182fa4f883e4cc28a127fdd70e652f8208d3c8dc4edc3bfc92fb49 0.0s
=> CANCELED [internal] load build context 0.2s
=> => transferring context: 3.74MB 0.1s
=> CACHED [linux/amd64 builder 1/3] FROM docker.io/library/eclipse-temurin:19-jre@sha256:e7934eaf29182fa4f883e4cc28a127fdd70e652f8208d3c8dc4edc3bfc92fb49 0.0s
=> => resolve docker.io/library/eclipse-temurin:19-jre@sha256:e7934eaf29182fa4f883e4cc28a127fdd70e652f8208d3c8dc4edc3bfc92fb49 0.0s
=> CACHED [linux/arm64 stage-1 1/7] FROM docker.io/library/eclipse-temurin:19-jre@sha256:e7934eaf29182fa4f883e4cc28a127fdd70e652f8208d3c8dc4edc3bfc92fb49 0.0s
=> => resolve docker.io/library/eclipse-temurin:19-jre@sha256:e7934eaf29182fa4f883e4cc28a127fdd70e652f8208d3c8dc4edc3bfc92fb49 0.0s
=> ERROR [linux/arm/v7 stage-1 2/7] RUN if [ "`uname -m`" = "x86_64" ]; then apt -y update && apt -y install git gcc && NONINTERACTIVE=1 /bin/bash -c "$(curl -fsSL https://raw.githubuserconten 0.1s
=> [linux/arm64 stage-1 2/7] RUN if [ "`uname -m`" = "x86_64" ]; then apt -y update && apt -y install git gcc && NONINTERACTIVE=1 /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/H 0.2s
=> CANCELED [linux/amd64 stage-1 2/7] RUN if [ "`uname -m`" = "x86_64" ]; then apt -y update && apt -y install git gcc && NONINTERACTIVE=1 /bin/bash -c "$(curl -fsSL https://raw.githubusercont 0.2s
------
> [linux/arm/v7 stage-1 2/7] RUN if [ "`uname -m`" = "x86_64" ]; then apt -y update && apt -y install git gcc && NONINTERACTIVE=1 /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" && eval "$(/home/linuxbrew/.linuxbrew/bin/brew shellenv)" && brew install jpeg-xl; fi:
0.117 exec /bin/sh: exec format error
------
WARNING: No output specified with docker-container driver. Build result will only remain in the build cache. To push result image into registry use --push or to load image into docker use --load
Dockerfile:8
--------------------
7 | # Install libjxl on x64 only. Homebrew is not available on other architectures.
8 | >>> RUN if [ "`uname -m`" = "x86_64" ]; then \
9 | >>> apt -y update && \
10 | >>> apt -y install git gcc && \
11 | >>> NONINTERACTIVE=1 /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" && \
12 | >>> eval "$(/home/linuxbrew/.linuxbrew/bin/brew shellenv)" && \
13 | >>> brew install jpeg-xl; fi
14 |
--------------------
ERROR: failed to solve: process "/bin/sh -c if [ \"`uname -m`\" = \"x86_64\" ]; then apt -y update && apt -y install git gcc && NONINTERACTIVE=1 /bin/bash -c \"$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)\" && eval \"$(/home/linuxbrew/.linuxbrew/bin/brew shellenv)\" && brew install jpeg-xl; fi" did not complete successfully: exit code: 1
### Additional info
_No response_
Same here. With version 0.11.2 I started seeing a bunch of ERROR: failed to solve: process "/bin/sh -c .... Reverted back to 0.11.0 and everything started working again
Same here. With version 0.11.2 I started seeing a bunch of
ERROR: failed to solve: process "/bin/sh -c .... Reverted back to 0.11.0 and everything started working again
I tried 0.11.0 and 0.10.5 but i still have the same problem. I tried to recreate the buildx builder on 0.10.5, but didn't work.
Could it be a problem with buildkit ? I can't find a way to pin the buildkit version in buildx though.
Looks like you forgot to install emulators: https://docs.docker.com/build/building/multi-platform/
$ docker run --privileged --rm tonistiigi/binfmt --install all
Looks like you forgot to install emulators: https://docs.docker.com/build/building/multi-platform/
$ docker run --privileged --rm tonistiigi/binfmt --install all
I will give it a spin, but the issue I had also happened on github actions with docker setup buildx
Also it used to work a few weeks ago.
but the issue I had also happened on github actions with docker setup buildx
Probably also forgot to add https://github.com/docker/setup-qemu-action
I will give it a spin
Let us know thanks
but the issue I had also happened on github actions with docker setup buildx
Probably also forgot to add docker/setup-qemu-action
I will give it a spin
Let us know thanks
it doesn't work, i still have the same issue
Oh actually the error is slightly different:
=> ERROR [linux/arm/v7 builder 3/3] RUN java -Djarmode=layertools -jar komga-1.3.0.jar extract 0.3s
=> CANCELED [linux/arm64 builder 2/3] COPY assembly/* / 0.0s
=> CANCELED [linux/amd64 builder 2/3] COPY assembly/* / 0.0s
=> CANCELED [linux/arm64 stage-1 2/6] WORKDIR app 0.0s
=> CANCELED [linux/amd64 stage-1 2/6] WORKDIR app 0.0s
------
> [linux/arm/v7 builder 3/3] RUN java -Djarmode=layertools -jar komga-1.3.0.jar extract:
0.218 runc run failed: unable to start container process: error during container init: error mounting "cgroup" to rootfs at "/sys/fs/cgroup": mount cgroup:/sys/fs/cgroup/openrc (via /proc/self/fd/6), flags: 0xf, data: openrc: invalid argument
------
WARNING: No output specified with docker-container driver. Build result will only remain in the build cache. To push result image into registry use --push or to load image into docker use --load
Dockerfile:4
--------------------
2 | ARG JAR=komga-1.3.0.jar
3 | COPY assembly/* /
4 | >>> RUN java -Djarmode=layertools -jar ${JAR} extract
5 |
6 | FROM eclipse-temurin:17-jre
--------------------
ERROR: failed to solve: process "/bin/sh -c java -Djarmode=layertools -jar ${JAR} extract" did not complete successfully: exit code: 1
unix:///Users/groebroeck/.colima/docker.sock
This looks related to https://github.com/abiosoft/colima/issues/720 as you're using colima.
unix:///Users/groebroeck/.colima/docker.sockThis looks related to abiosoft/colima#720 as you're using colima.
thanks, i will have a look.
However i also have the issue on Github Actions: https://github.com/gotson/komga/actions/runs/5746101884/job/15575224773
I did setup QEMU and buildx using the docker actions, and the failure happens during the JReleaser full release step.
I did setup QEMU and buildx using the docker actions, and the failure happens during the JReleaser full release step.
Hum I don't see any error in this run:
Neither any usage of our build-push-action.
Ah I see the error in https://github.com/gotson/komga/actions/runs/5746101884/job/15575224773#step:11:577
But doesn't look like this gradle action propagates errors properly to the workflow.
Also I don't see what gradle invokes as build command. I guess it just invokes docker build instead of docker buildx build so it would not use the container builder. If that's the case you need to set install: true in the setup-buildx-action: https://github.com/docker/setup-buildx-action#inputs
But doesn't look like this gradle action propagates errors properly to the workflow.
that's by design with JReleaser, i don't want the whole workflow to fail when docker fails to build/push
Here is the log of what command was invoked:
[DEBUG] [docker] docker -l error buildx build --platform linux/amd64,linux/arm/v7,linux/arm64/v8 --cache-from gotson/komga:latest --quiet --file /home/runner/work/komga/komga/komga/build/jreleaser/package/komga/docker/Dockerfile --tag docker.io/gotson/komga:latest --tag docker.io/gotson/komga:1.3.0 --tag docker.io/gotson/komga:1.x --tag ghcr.io/gotson/komga:latest --tag ghcr.io/gotson/komga:1.3.0 --tag ghcr.io/gotson/komga:1.x /home/runner/work/komga/komga/komga/build/jreleaser/package/komga/docker
[ERROR] [docker] Dockerfile:8
[ERROR] [docker] --------------------
[ERROR] [docker] 7 | # Install libjxl on x64 only. Homebrew is not available on other architectures.
[ERROR] [docker] 8 | >>> RUN if [ "`uname -m`" = "x86_64" ]; then \
[ERROR] [docker] 9 | >>> apt -y update && \
[ERROR] [docker] 10 | >>> apt -y install git gcc && \
[ERROR] [docker] 11 | >>> NONINTERACTIVE=1 /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" && \
[ERROR] [docker] 12 | >>> eval "$(/home/linuxbrew/.linuxbrew/bin/brew shellenv)" && \
[ERROR] [docker] 13 | >>> brew install jpeg-xl; fi
[ERROR] [docker] 14 |
[ERROR] [docker] --------------------
[ERROR] [docker] ERROR: failed to solve: process "/bin/sh -c if [ \"`uname -m`\" = \"x86_64\" ]; then apt -y update && apt -y install git gcc && NONINTERACTIVE=1 /bin/bash -c \"$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)\" && eval \"$(/home/linuxbrew/.linuxbrew/bin/brew shellenv)\" && brew install jpeg-xl; fi" did not complete successfully: exit code: 1
Logs look truncated. Would need full logs (before Dockerfile:8 like https://github.com/docker/buildx/issues/1986#issuecomment-1665158062).
unfortunately that's all JReleaser outputs, here is the full trace.log but it doesn't show more on the execution of the external docker command, probably because of the --quiet flag
trace.log
I am reproducing on another Macbook which has Docker Desktop installed.
Here is the log from docker buildx build --platform linux/amd64,linux/arm/v7,linux/arm64/v8 --no-cache --file Dockerfile .
I also attach some details for this machine:
Docker version:
Client:
Cloud integration: v1.0.33
Version: 24.0.2
API version: 1.43
Go version: go1.20.4
Git commit: cb74dfc
Built: Thu May 25 21:51:16 2023
OS/Arch: darwin/amd64
Context: desktop-linux
Server: Docker Desktop 4.20.1 (110738)
Engine:
Version: 24.0.2
API version: 1.43 (minimum version 1.12)
Go version: go1.20.4
Git commit: 659604f
Built: Thu May 25 21:52:17 2023
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.6.21
GitCommit: 3dce8eb055cbb6872793272b4f20ed16117344f8
runc:
Version: 1.1.7
GitCommit: v1.1.7-0-g860f061
docker-init:
Version: 0.19.0
GitCommit: de40ad0
docker buildx version:
github.com/docker/buildx v0.10.5 86bdced7766639d56baa4c7c449a4f6468490f87
docker buildx ls
NAME/NODE DRIVER/ENDPOINT STATUS BUILDKIT PLATFORMS
festive_liskov * docker-container
festive_liskov0 desktop-linux running v0.12.1 linux/amd64, linux/amd64/v2, linux/arm64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6
frosty_newton docker-container
frosty_newton0 desktop-linux stopped
sharp_kilby docker-container
sharp_kilby0 unix:///var/run/docker.sock stopped
default docker
default default running v0.11.7-0.20230525183624-798ad6b0ce9f linux/amd64, linux/amd64/v2, linux/arm64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6
desktop-linux docker
desktop-linux desktop-linux running v0.11.7-0.20230525183624-798ad6b0ce9f linux/amd64, linux/amd64/v2, linux/arm64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6
Doesn't seem to be the same issue:
#15 37.41 ==> Checking for `sudo` access (which may request your password)...
#15 37.41 Don't run this as root!
This docker file works perfectly fine with docker build but fails with docker buildx build, for me that's the concern.
I have the same issue with a M2
Same issue with M1 Mac confirmed. Are you using colima @JorTurFer ?
There is issue related open in colima about cgroup mode. A week earlier I ran into issue running KinD with colima that was also related to cgroup mounting and similar runc errors followed.
Issue thread is here: https://github.com/abiosoft/colima/issues/720
If you are on colima, a solution is to use rootless buildx builder:
docker buildx create \
--name fixed_builder \
--driver-opt 'image=moby/buildkit:v0.12.1-rootless' \
--bootstrap --use
Before running the above command, can remove the older buildx runners with docker buildx rm ...
Solution from here: https://github.com/abiosoft/colima/issues/764#issuecomment-1670603248
Are you using colima @JorTurFer ?
Yes, I do use colima. Thanks for the workaround, I really appreciated it 🙇
Looks like you forgot to install emulators: https://docs.docker.com/build/building/multi-platform/
$ docker run --privileged --rm tonistiigi/binfmt --install all
I had the same issue with M1 Mac and this one solved my problem! Thanks!
Hi - I'm running into this issue right now, trying to only build it with Github Actions (ubuntu-latest). It was working last year.
I updated the QEMU and all the other actions to the latest, but it's still not working. Not using colima or anything like that, and none of the fixes proposed in this thread have worked.
Specifically, failing with
#14 [linux/amd64 6/11] RUN echo "*** Install FFF ***" && set -x && if [ -z 4.30.0 ]; then echo "FFF Using Default Release"; python3 -m pip --no-cache-dir install FanFicFare; else echo "FF Using 4.30.0 Release"; python3 -m pip --no-cache-dir install --extra-index-url https://testpypi.python.org/pypi FanFicFare==4.30.0; fi
#14 0.213 runc run failed: unable to start container process: exec: "/bin/sh": stat /bin/sh: no such file or directory
#14 ERROR: process "/bin/sh -c echo \"*** Install FFF ***\" && \tset -x && if [ -z ${FFF_RELEASE} ]; then \t\techo \"FFF Using Default Release\"; python3 -m pip --no-cache-dir install FanFicFare; else \t\techo \"FF Using ${FFF_RELEASE} Release\"; python3 -m pip --no-cache-dir install --extra-index-url https://testpypi.python.org/pypi FanFicFare==${FFF_RELEASE}; fi" did not complete successfully: exit code: 1
#9 [linux/arm64 2/11] RUN set -x && apt-get update && apt-get install -y --upgrade bash ca-certificates gcc wget xdg-utils curl dbus jq python3
#9 CANCELED
------
> [linux/amd64 6/11] RUN echo "*** Install FFF ***" && set -x && if [ -z 4.30.0 ]; then echo "FFF Using Default Release"; python3 -m pip --no-cache-dir install FanFicFare; else echo "FF Using 4.30.0 Release"; python3 -m pip --no-cache-dir install --extra-index-url https://testpypi.python.org/pypi FanFicFare==4.30.0; fi:
0.213 runc run failed: unable to start container process: exec: "/bin/sh": stat /bin/sh: no such file or directory
------
Dockerfile:56
--------------------
55 |
56 | >>> RUN echo "*** Install FFF ***" && \
57 | >>> set -x && \
58 | >>> if [ -z ${FFF_RELEASE} ]; then \
59 | >>> echo "FFF Using Default Release"; \
60 | >>> python3 -m pip --no-cache-dir install FanFicFare; \
61 | >>> else \
62 | >>> echo "FF Using ${FFF_RELEASE} Release"; \
63 | >>> python3 -m pip --no-cache-dir install --extra-index-url https://testpypi.python.org/pypi FanFicFare==${FFF_RELEASE}; \
64 | >>> fi
65 | RUN echo "*** Install Other Python Packages ***" && \
--------------------
ERROR: failed to solve: process "/bin/sh -c echo \"*** Install FFF ***\" && \tset -x && if [ -z ${FFF_RELEASE} ]; then \t\techo \"FFF Using Default Release\"; python3 -m pip --no-cache-dir install FanFicFare; else \t\techo \"FF Using ${FFF_RELEASE} Release\"; python3 -m pip --no-cache-dir install --extra-index-url https://testpypi.python.org/pypi FanFicFare==${FFF_RELEASE}; fi" did not complete successfully: exit code: 1
Error: buildx failed with: ERROR: failed to solve: process "/bin/sh -c echo \"*** Install FFF ***\" && \tset -x && if [ -z ${FFF_RELEASE} ]; then \t\techo \"FFF Using Default Release\"; python3 -m pip --no-cache-dir install FanFicFare; else \t\techo \"FF Using ${FFF_RELEASE} Release\"; python3 -m pip --no-cache-dir install --extra-index-url https://testpypi.python.org/pypi FanFicFare==${FFF_RELEASE}; fi" did not complete successfully: exit code: 1
Thanks in advance.
$ docker run --privileged --rm tonistiigi/binfmt --install all
We had same issue on Windows10 + WSL(Ubuntu22.04) and this works. Thanks!
I have the same problem. RUN git\your_project> docker pull python:3.11.1-slim
And change first string in your docker file: FROM python:3.11.1-alpine3.17 -> FROM python:3.11.1-slim
That problem comes from the restrictions of your virtual mashine
might also happen if you try to install without -y in dockerfile:
RUN apt install -y whatever
Seeing this for a multi-platform build in CI on an amd64 runner with qemu. I can build one image just fine for amd64 and arm64, but the other one fails at the first RUN with
exec /bin/sh: no such file or directory
Seeing this for a multi-platform build in CI on an amd64 runner with qemu. I can build one image just fine for amd64 and arm64, but the other one fails at the first
RUNwith
Same.
