buildkit Docker Build failing if Target is set (`stat /bin/sh: no such file or directory`)

Docker Build failing if Target is set (`stat /bin/sh: no such file or directory`)

Open chris-beedie opened this issue 5 months ago • 4 comments

Contributing guidelines and issue reporting guide

[x] I've read the contributing guidelines and wholeheartedly agree. I've also read the issue reporting guide.

Well-formed report checklist

[x] I have found a bug that the documentation does not mention anything about my problem
[x] I have found a bug that there are no open or closed issues that are related to my problem
[x] I have provided version/information about my environment and done my best to provide a reproducer

Description of bug

Bug description

This issue seems specific to my work dev box, an AWS EC2 running Ubuntu 24.04.2. I have been unable to replicate elsewhere.

I get very inconsistent behaviour when building an image with a target specified. It fails on seemingly arbitrary lines.

Disabling buildkit does resolve the issue, but I would like to avoid this if at all possible. Using the vfs storage driver also seems to fix it, but performance is dreadful.

Originally posted to Stack Overflow and recommended to raise a bug.

Reproduction

Running:

docker build --debug --progress=plain --no-cache -t api-maint --target maint .

With this mock Dockerfile:

FROM python:3.12-slim-bookworm AS base

RUN pwd
RUN pwd
RUN pwd
RUN pwd
RUN pwd
RUN pwd
RUN pwd
RUN pwd

FROM base AS dev

RUN pwd
RUN pwd
RUN pwd
RUN pwd
RUN pwd
RUN pwd
RUN pwd
RUN pwd

FROM dev AS maint


RUN pwd
RUN pwd
RUN pwd
RUN pwd
RUN pwd
RUN pwd
RUN pwd
RUN pwd
RUN pwd
RUN pwd
RUN pwd
RUN pwd
RUN pwd
RUN pwd

CMD []

I get the error:

#10 [base 7/9] RUN pwd
#10 0.317 runc run failed: unable to start container process: error during container init: exec: "/bin/sh": stat /bin/sh: no such file or directory
#10 ERROR: process "/bin/sh -c pwd" did not complete successfully: exit code: 1
------
> [base 7/9] RUN pwd:
0.317 runc run failed: unable to start container process: error during container init: exec: "/bin/sh": stat /bin/sh: no such file or directory
------
Dockerfile:8
--------------------
   6 |     RUN pwd
   7 |     RUN pwd
   8 | >>> RUN pwd
   9 |     RUN pwd
  10 |     RUN pwd
--------------------
ERROR: failed to solve: process "/bin/sh -c pwd" did not complete successfully: exit code: 1
1143  /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
github.com/moby/buildkit/executor/runcexecutor.exitError
        /root/build-deb/engine/vendor/github.com/moby/buildkit/executor/runcexecutor/executor.go:391
github.com/moby/buildkit/executor/runcexecutor.(*runcExecutor).Run
        /root/build-deb/engine/vendor/github.com/moby/buildkit/executor/runcexecutor/executor.go:339
github.com/moby/buildkit/solver/llbsolver/ops.(*ExecOp).Exec
        /root/build-deb/engine/vendor/github.com/moby/buildkit/solver/llbsolver/ops/exec.go:489
github.com/moby/buildkit/solver.(*sharedOp).Exec.func2
        /root/build-deb/engine/vendor/github.com/moby/buildkit/solver/jobs.go:1106
github.com/moby/buildkit/util/flightcontrol.(*call[...]).run
        /root/build-deb/engine/vendor/github.com/moby/buildkit/util/flightcontrol/flightcontrol.go:123
sync.(*Once).doSlow
        /usr/local/go/src/sync/once.go:76
sync.(*Once).Do
        /usr/local/go/src/sync/once.go:67
runtime.goexit
        /usr/local/go/src/runtime/asm_amd64.s:1700

63863 v0.23.0 /usr/libexec/docker/cli-plugins/docker-buildx buildx build --debug --progress=plain --no-cache -t api-maint --target maint .
google.golang.org/grpc.(*ClientConn).Invoke
        google.golang.org/[email protected]/call.go:35
github.com/moby/buildkit/api/services/control.(*controlClient).Solve
        github.com/moby/[email protected]/api/services/control/control_grpc.pb.go:88
github.com/moby/buildkit/client.(*Client).solve.func2
        github.com/moby/[email protected]/client/solve.go:268
golang.org/x/sync/errgroup.(*Group).Go.func1
        golang.org/x/[email protected]/errgroup/errgroup.go:79
runtime.goexit
        runtime/asm_amd64.s:1700

1143  /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
github.com/moby/buildkit/solver.(*edge).execOp
        /root/build-deb/engine/vendor/github.com/moby/buildkit/solver/edge.go:963
github.com/moby/buildkit/solver/internal/pipe.NewWithFunction[...].func2
        /root/build-deb/engine/vendor/github.com/moby/buildkit/solver/internal/pipe/pipe.go:78
runtime.goexit
        /usr/local/go/src/runtime/asm_amd64.s:1700

63863 v0.23.0 /usr/libexec/docker/cli-plugins/docker-buildx buildx build --debug --progress=plain --no-cache -t api-maint --target maint .
github.com/moby/buildkit/client.(*Client).solve.func2
        github.com/moby/[email protected]/client/solve.go:285
golang.org/x/sync/errgroup.(*Group).Go.func1
        golang.org/x/[email protected]/errgroup/errgroup.go:79

1143  /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
github.com/moby/buildkit/solver/llbsolver/ops.(*ExecOp).Exec
        /root/build-deb/engine/vendor/github.com/moby/buildkit/solver/llbsolver/ops/exec.go:510
github.com/moby/buildkit/solver.(*sharedOp).Exec.func2
        /root/build-deb/engine/vendor/github.com/moby/buildkit/solver/jobs.go:1106

Changes to the Dockerfile above (and using my real one rather than the mock) all break in the same way, but on different lines. Occasionally something will build, but any minor change, like introducing a new line and it will break again.

Version information

docker buildx version && docker buildx inspect:

github.com/docker/buildx v0.23.0 28c90ea
Name:          default
Driver:        docker
Last Activity: 2025-05-02 09:44:52 +0000 UTC

Nodes:
Name:             default
Endpoint:         default
Status:           running
BuildKit version: v0.21.0
Platforms:        linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/amd64/v4, linux/386
Labels:
org.mobyproject.buildkit.worker.moby.host-gateway-ip: 172.17.0.1

Docker Info:

Client: Docker Engine - Community
Version:    28.1.1
Context:    default
Debug Mode: false
Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.23.0
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.35.1
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
Images: 0
Server Version: 28.1.1
Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
Logging Driver: json-file
Cgroup Driver: systemd
Cgroup Version: 2
Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 05044ec0a9a75232cad458027ca83437aae3f4da
runc version: v1.2.5-0-g59923ef
init version: de40ad0
Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
Kernel Version: 6.8.0-1027-aws
Operating System: Ubuntu 24.04.2 LTS
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 7.638GiB
Name: XXXXXX
ID: b297b9e4-9f73-4367-b055-99f34b95711c
Docker Root Dir: /var/lib/docker
Debug Mode: false
Experimental: false
Insecure Registries:
  ::1/128
  127.0.0.0/8
Live Restore Enabled: false

Thanks

May 02 '25 09:05 chris-beedie

buildkit buildkit copied to clipboard

Docker Build failing if Target is set (`stat /bin/sh: no such file or directory`)

Contributing guidelines and issue reporting guide

Well-formed report checklist

Description of bug

Bug description

Reproduction

Version information

buildkit
buildkit copied to clipboard