cli icon indicating copy to clipboard operation
cli copied to clipboard

Docker cant find local images on macbook m1

Open marcusgraf opened this issue 4 years ago • 28 comments

Description

When building an image with a FROM local-image:latest statement with local-image being present (in my case offerplace-shared:latest), Docker anyway tries to pull that image from docker.io and fails. Since there seem to be any issues with image names that contain a dash, I also tried to replace that part with a repository name: offerplace/shared:latest, but to no avail - it still tries to pull it.

Steps to reproduce the issue:

  1. Build image local/image1
  2. Build image local/image2, with a FROM local/image1:latest statement
  3. Watch it failing

Describe the results you received:

Docker fails with the following error on my machine:

#1 [internal] load build definition from Dockerfile
#1 sha256:bb8fff504e2f268680b3eb7591394abfda1b2584b1e909984c56f0d067f89866
#1 transferring dockerfile: 511B done
#1 DONE 0.0s
#2 [internal] load .dockerignore
#2 sha256:f8cf604e196078a17ce397b978e6abb30bc09501f5d3094cfd91ee619b2e8483
#2 transferring context: 2B done
#2 DONE 0.0s
#3 [internal] load metadata for docker.io/library/openjdk:11-jre-slim
#3 sha256:0a99f1463278d5e63857a5e8637c92992bf2cd37d95a42501fa5494f1d624517
#3 CANCELED
#4 [internal] load metadata for docker.io/offerplace/shared:latest
#4 sha256:38ed34553b1dcac3fdf787bb8f99802fc2940aced9fbdcdf41bfab9b3f7f8bfd
#4 ERROR: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed
------
 > [internal] load metadata for docker.io/offerplace/shared:latest:
------
failed to solve with frontend dockerfile.v0: failed to create LLB definition: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

Describe the results you expected:

The build does not fail, since the image exists locally.

Additional information you deem important (e.g. issue happens only occasionally):

The issue arised suddenly and permanent without (at least as I remember) changing things. Even previously successfully built images cannot be built.

Output of docker version:

Client:
 Cloud integration: 1.0.17
 Version:           20.10.8
 API version:       1.41
 Go version:        go1.16.6
 Git commit:        3967b7d
 Built:             Fri Jul 30 19:55:20 2021
 OS/Arch:           darwin/arm64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.8
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.16.6
  Git commit:       75249d8
  Built:            Fri Jul 30 19:53:34 2021
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.4.9
  GitCommit:        e25210fe30a0a703442421b0f60afac609f950a3
 runc:
  Version:          1.0.1
  GitCommit:        v1.0.1-0-g4144b63
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Output of docker info:

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Build with BuildKit (Docker Inc., v0.6.1-docker)
  compose: Docker Compose (Docker Inc., v2.0.0-rc.2)
  scan: Docker Scan (Docker Inc., v0.8.0)

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 3
 Server Version: 20.10.8
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: e25210fe30a0a703442421b0f60afac609f950a3
 runc version: v1.0.1-0-g4144b63
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 5.10.47-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 4
 Total Memory: 1.942GiB
 Name: docker-desktop
 ID: DNCI:7JKR:A2UM:CRJ5:SFVA:7GNU:6NKE:4CEC:I3IB:Y5U2:W4TD:W74C
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Additional environment details (AWS, VirtualBox, physical, etc.): MacBook Pro (13-inch, M1, 2020) macOS Big Sur 11.3.1 Processor Apple M1

marcusgraf avatar Sep 03 '21 14:09 marcusgraf

looks like same as this issue: https://forums.docker.com/t/not-finding-local-unprefixed-docker-image-in-the-local-cache/113297

Doesn't look like there's been a resolution yet.

joshspicer avatar Sep 08 '21 20:09 joshspicer

What does docker buildx ls show for both of you?

What I think might be the case is that the default builder is set to a builder that uses the container driver. In this case, builds are executed in a containerised builder, which does not have direct access to local images in the docker daemon's image cache. Does it work after you run docker buildx use default ?

/cc @crazy-max

thaJeztah avatar Sep 09 '21 15:09 thaJeztah

Thanks for the response, docker buildx ls is showing me that:

NAME/NODE       DRIVER/ENDPOINT STATUS  PLATFORMS
desktop-linux   docker
  desktop-linux desktop-linux   running linux/arm64, linux/amd64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/arm/v7, linux/arm/v6
default *       docker
  default       default         running linux/arm64, linux/amd64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/arm/v7, linux/arm/v6

And looks like docker buildx use default is not solving the issue

marcusgraf avatar Sep 13 '21 07:09 marcusgraf

Mine was also set to default, and trying to change it didn't help

joshspicer avatar Sep 16 '21 17:09 joshspicer

According to this, the following solve the issue.

export DOCKER_BUILDKIT=0
export COMPOSE_DOCKER_CLI_BUILD=0

pkhadka56 avatar Nov 01 '21 10:11 pkhadka56

I think this may be fixed in docker 20.10.10 through https://github.com/moby/moby/pull/42954 (and https://github.com/moby/moby/pull/42951), which will be in an upcoming release of Docker Desktop.

/cc @crazy-max ^

thaJeztah avatar Nov 02 '21 11:11 thaJeztah

If that's correct, then instead of disabling buildkit, I would recommend using --platform=linux/arm64 on docker build as a workaround; see https://github.com/moby/moby/issues/42893#issuecomment-943924990

thaJeztah avatar Nov 02 '21 11:11 thaJeztah

Still seeing this issue on Docker version 20.10.10, build b485636

swills avatar Nov 17 '21 16:11 swills

@swills I can't repro anymore since 20.10.10:

# ./Dockerfile.foo
FROM alpine
RUN echo "Hello foo!"
# ./Dockerfile.bar
FROM local/foo-parent
RUN echo "Hello bar!"
DOCKER_BUILDKIT=1 docker build -f Dockerfile.foo -t local/foo-parent .
DOCKER_BUILDKIT=1 docker build -f Dockerfile.bar -t local/foo-child .

Can you post a repro please?

crazy-max avatar Nov 17 '21 16:11 crazy-max

@crazy-max I'm having the same problem.

$ docker -v
Docker version 20.10.10, build b485636

$ docker-compose -v
Docker Compose version v2.1.

Running it with the docker command described in the comments succeeded, but running it via docker-compose failed.

# docker-compose.yml

version: "3.8"

services:
  foo:
    build:
      context: .
      dockerfile: Dockerfile.foo
    image: foo
  bar:
    build:
      context: .
      dockerfile: Dockerfile.bar
$ docker-compose build
[+] Building 7.4s (6/6) FINISHED
 => [tes_bar internal] load build definition from Dockerfile.bar                                                   0.0s
 => => transferring dockerfile: 72B                                                                                0.0s
 => [foo internal] load build definition from Dockerfile.foo                                                       0.0s
 => => transferring dockerfile: 75B                                                                                0.0s
 => [tes_bar internal] load .dockerignore                                                                          0.0s
 => => transferring context: 2B                                                                                    0.0s
 => [foo internal] load .dockerignore                                                                              0.0s
 => => transferring context: 2B                                                                                    0.0s
 => ERROR [tes_bar internal] load metadata for docker.io/library/foo:latest                                        7.3s
 => CANCELED [foo internal] load metadata for docker.io/library/alpine:latest                                      7.3s
------
 > [tes_bar internal] load metadata for docker.io/library/foo:latest:
------
failed to solve: rpc error: code = Unknown desc = failed to solve with frontend dockerfile.v0: failed to create LLB definition: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

Oddly enough, commenting out bar from docker-compose.yml and building it succeeds, and uncommenting it again and building it succeeds in execution.

$ docker-compose build
[+] Building 0.9s (6/6) FINISHED
 => [internal] load build definition from Dockerfile.foo                                                           0.0s
 => => transferring dockerfile: 75B                                                                                0.0s
 => [internal] load .dockerignore                                                                                  0.0s
 => => transferring context: 2B                                                                                    0.0s
 => [internal] load metadata for docker.io/library/alpine:latest                                                   0.8s
 => [1/2] FROM docker.io/library/alpine@sha256:635f0aa53d99017b38d1a0aa5b2082f7812b03e3cdb299103fe77b5c8a07f1d2    0.0s
 => CACHED [2/2] RUN echo "Hello foo!"                                                                             0.0s
 => exporting to image                                                                                             0.0s
 => => exporting layers                                                                                            0.0s
 => => writing image sha256:fad9a919f1392db57741521b9a151cd2796b675263aee53bb5787c75a1875627                       0.0s
 => => naming to docker.io/library/foo                                                                             0.0s
 
 $ docker-compose build
[+] Building 0.4s (11/11) FINISHED
 => [tes_bar internal] load build definition from Dockerfile.bar                                                   0.0s
 => => transferring dockerfile: 72B                                                                                0.0s
 => [foo internal] load build definition from Dockerfile.foo                                                       0.0s
 => => transferring dockerfile: 35B                                                                                0.0s
 => [tes_bar internal] load .dockerignore                                                                          0.0s
 => => transferring context: 2B                                                                                    0.0s
 => [foo internal] load .dockerignore                                                                              0.0s
 => => transferring context: 2B                                                                                    0.0s
 => [tes_bar internal] load metadata for docker.io/library/foo:latest                                              0.0s
 => [tes_bar 1/2] FROM docker.io/library/foo                                                                       0.0s
 => [tes_bar 2/2] RUN echo "Hello bar!"                                                                            0.2s
 => [foo internal] load metadata for docker.io/library/alpine:latest                                               0.3s
 => [foo] exporting to image                                                                                       0.0s
 => => exporting layers                                                                                            0.0s
 => => writing image sha256:ce46d475f89366053d5c6416d814dc18e12b5f550152e3ec5c07c0c88eb9d3c7                       0.0s
 => => naming to docker.io/library/tes_bar                                                                         0.0s
 => => writing image sha256:fad9a919f1392db57741521b9a151cd2796b675263aee53bb5787c75a1875627                       0.0s
 => => naming to docker.io/library/foo                                                                             0.0s
 => [foo 1/2] FROM docker.io/library/alpine@sha256:635f0aa53d99017b38d1a0aa5b2082f7812b03e3cdb299103fe77b5c8a07f1  0.0s
 => CACHED [foo 2/2] RUN echo "Hello foo!"

naomichi-y avatar Nov 22 '21 19:11 naomichi-y

@naomichi-y

Oddly enough, commenting out bar from docker-compose.yml and building it succeeds, and uncommenting it again and building it succeeds in execution.

Not linked to this issue but I would guess that's because your bar service does not depend on foo service so it builds both and bar fails because foo image is not yet built.

crazy-max avatar Nov 22 '21 19:11 crazy-max

@crazy-max Thanks, I tried adding depends_on to bar, but it doesn't seem to change. The same setting works fine in Intel machine.

naomichi-y avatar Nov 22 '21 19:11 naomichi-y

Able to reproduce this issue on Macbook Pro (M1 Pro chip):

➜  Docker docker images | head -n 2
REPOSITORY            TAG          IMAGE ID       CREATED        SIZE
node-serve            latest       46d92a2c8716   10 hours ago   118MB

➜  Docker cat Dockerfile 
FROM node-serve:latest
WORKDIR /data

➜  Docker docker build -t test_local_build:latest .
[+] Building 17.5s (3/3) FINISHED                                                              
 => [internal] load build definition from Dockerfile                                      0.0s
 => => transferring dockerfile: 79B                                                       0.0s
 => [internal] load .dockerignore                                                         0.0s
 => => transferring context: 2B                                                           0.0s
 => ERROR [internal] load metadata for docker.io/library/node-serve:latest               17.4s
------
 > [internal] load metadata for docker.io/library/node-serve:latest:
------
failed to solve with frontend dockerfile.v0: failed to create LLB definition: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

➜  Docker docker build --platform linux/arm64 -t test_local_build:latest .
[+] Building 17.6s (3/3) FINISHED                                                              
 => [internal] load build definition from Dockerfile                                      0.0s
 => => transferring dockerfile: 36B                                                       0.0s
 => [internal] load .dockerignore                                                         0.0s
 => => transferring context: 2B                                                           0.0s
 => ERROR [internal] load metadata for docker.io/library/node-serve:latest               17.5s
------
 > [internal] load metadata for docker.io/library/node-serve:latest:
------
failed to solve with frontend dockerfile.v0: failed to create LLB definition: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

Im my case, the image node-serve:latest was created locally by committing a container. and

➜  Docker docker inspect --format='{{.Os}}/{{.Architecture}}' node-serve:latest
linux/arm64

hywhuangyuwei avatar Nov 24 '21 21:11 hywhuangyuwei

Able to reproduce this issue on Macbook Pro (M1 Pro chip):

➜  Docker docker images | head -n 2
REPOSITORY            TAG          IMAGE ID       CREATED        SIZE
node-serve            latest       46d92a2c8716   10 hours ago   118MB

➜  Docker cat Dockerfile 
FROM node-serve:latest
WORKDIR /data

➜  Docker docker build -t test_local_build:latest .
[+] Building 17.5s (3/3) FINISHED                                                              
 => [internal] load build definition from Dockerfile                                      0.0s
 => => transferring dockerfile: 79B                                                       0.0s
 => [internal] load .dockerignore                                                         0.0s
 => => transferring context: 2B                                                           0.0s
 => ERROR [internal] load metadata for docker.io/library/node-serve:latest               17.4s
------
 > [internal] load metadata for docker.io/library/node-serve:latest:
------
failed to solve with frontend dockerfile.v0: failed to create LLB definition: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

➜  Docker docker build --platform linux/arm64 -t test_local_build:latest .
[+] Building 17.6s (3/3) FINISHED                                                              
 => [internal] load build definition from Dockerfile                                      0.0s
 => => transferring dockerfile: 36B                                                       0.0s
 => [internal] load .dockerignore                                                         0.0s
 => => transferring context: 2B                                                           0.0s
 => ERROR [internal] load metadata for docker.io/library/node-serve:latest               17.5s
------
 > [internal] load metadata for docker.io/library/node-serve:latest:
------
failed to solve with frontend dockerfile.v0: failed to create LLB definition: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

Im my case, the image node-serve:latest was created locally by committing a container. and

➜  Docker docker inspect --format='{{.Os}}/{{.Architecture}}' node-serve:latest
linux/arm64

The same issue for: FROM --platform=linux/arm64 node-serve:latest

hywhuangyuwei avatar Nov 24 '21 21:11 hywhuangyuwei

FROM --platform=linux/arm64 node-serve:latest

Report: After upgraded from 20.10.8 to 20.10.10 docker build --platform linux/arm64 -t xxx . builds for me.

hywhuangyuwei avatar Nov 25 '21 07:11 hywhuangyuwei

This issue is extremely frustrating. I've created custom builder docker buildx create --name amd64_builder --platform linux/amd64 to compile docker images GCP/AWS.

And when I run DOCKER_BUILDKIT=0 docker buildx bake --load --builder amd64_builder -f dc.base.yml --no-cache

the resulting image is nevertheless

 "Architecture": "arm64",
        "Variant": "v8",
        "Os": "linux",

Update: Had to include the following in my yaml file

x-bake:
       platforms:
         - linux/amd64

skol101 avatar Dec 16 '21 18:12 skol101

I was initially wondering if this was related to https://github.com/moby/moby/pull/43103 (i.e., if loading the built image into the docker image cache would omit the platform).

I don't think that's related though. I think the issue here may be that, while the custom builder is created with --platform, it's still a local builder, only using the docker-container driver, which means that a buildkit instance is started running inside a container. That container won't have linux/amd64 as native platform (if your machine is running arm64), so other architectures would still be built using emulation (QEMU).

Here's what a custom builder shows on my own setup (which is the reverse, as I'm on x86, so I created a builder with --platform=linux/arm64);

docker buildx create --name arm64_builder --platform linux/arm64

docker buildx inspect arm64_builder
Name:   arm64_builder
Driver: docker-container

Nodes:
Name:      arm64_builder0
Endpoint:  unix:///var/run/docker.sock
Status:    running
Platforms: linux/arm64*, linux/amd64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6

So what I suspect happens is that (given that no desired platform is specified), the builder will use the native architecture (linux/arm64) as default, and won't use the emulated (amd64) architecture, unless specified.

That said, the documentation is a bit ambiguous here https://docs.docker.com/engine/reference/commandline/buildx_create/#platform

The --platform flag sets the platforms supported by the node. It expects a comma-separated list of platforms of the form OS/architecture/variant. The node will also automatically detect the platforms it supports, but manual values take priority over the detected ones and can be used when multiple nodes support building for the same platform.

It describes that the specified platform should be preferred, but it also detects other supported arches. It's a bit unclear what the "preferred" here means;

  • use as default if no --platform is specified during build
  • if multiple architectures are specified on build, pick this builder as the preferred builder for the specified platform?

/cc @docker/build

thaJeztah avatar Jan 03 '22 14:01 thaJeztah

^^ I discussed the expected behavior with @crazy-max, and what it comes down to is;

  • the --platform option on docker buildx create is used for situations where a pool of builders is available. So, for example, if I have a "ppc64le" builder and an "arm64" builder, and I start a docker build --platform=linux/ppc64le,linux/arm64, then the builds will be schedules such that each of those architectures will be performed on the "best match" (which will be matched by the native architecture by default, but can be influenced by the --platform option that was used when creating the builder).
  • what it does NOT do is override the default platform if no platform is specified on docker build (nor FROM --platform). If no platform is specified for a build then the default will always be to build for the native platform (in your case, that would be arm64).

The documentation could definitely be improved to better describe this, but we're also discussing what would be the "least surprising" behavior for this case;

  • use the --platform that was specified when creating the builder "as if it was the native platform" (so default to that platform, even if that means emulation)
  • print a warning that no platform was specified for the build, and that the build won't match the platform that was specified when creating the builder
  • same as above, but produce an error

I think part of the confusion is also that all "supported" architectures are added to the builder, even though only a single platform was specified when creating the builder. Possible options for that could be to only list the specified platform (ignore any other architecture that's supported), which would require the user to always provide the full list of architectures that the builder can be used for (in order of preference), which may be a bit awkward as well.

thaJeztah avatar Jan 03 '22 16:01 thaJeztah

Exercise network problem

Nimesh199398 avatar Jan 28 '22 08:01 Nimesh199398

Description

When building an image with a FROM local-image:latest statement with local-image being present (in my case offerplace-shared:latest), Docker anyway tries to pull that image from docker.io and fails. Since there seem to be any issues with image names that contain a dash, I also tried to replace that part with a repository name: offerplace/shared:latest, but to no avail - it still tries to pull it.

Steps to reproduce the issue:

  1. Build image local/image1
  2. Build image local/image2, with a FROM local/image1:latest statement
  3. Watch it failing

Describe the results you received:

Docker fails with the following error on my machine:

#1 [internal] load build definition from Dockerfile
#1 sha256:bb8fff504e2f268680b3eb7591394abfda1b2584b1e909984c56f0d067f89866
#1 transferring dockerfile: 511B done
#1 DONE 0.0s
#2 [internal] load .dockerignore
#2 sha256:f8cf604e196078a17ce397b978e6abb30bc09501f5d3094cfd91ee619b2e8483
#2 transferring context: 2B done
#2 DONE 0.0s
#3 [internal] load metadata for docker.io/library/openjdk:11-jre-slim
#3 sha256:0a99f1463278d5e63857a5e8637c92992bf2cd37d95a42501fa5494f1d624517
#3 CANCELED
#4 [internal] load metadata for docker.io/offerplace/shared:latest
#4 sha256:38ed34553b1dcac3fdf787bb8f99802fc2940aced9fbdcdf41bfab9b3f7f8bfd
#4 ERROR: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed
------
 > [internal] load metadata for docker.io/offerplace/shared:latest:
------
failed to solve with frontend dockerfile.v0: failed to create LLB definition: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

Describe the results you expected:

The build does not fail, since the image exists locally.

Additional information you deem important (e.g. issue happens only occasionally):

The issue arised suddenly and permanent without (at least as I remember) changing things. Even previously successfully built images cannot be built.

Output of docker version:

Client:
 Cloud integration: 1.0.17
 Version:           20.10.8
 API version:       1.41
 Go version:        go1.16.6
 Git commit:        3967b7d
 Built:             Fri Jul 30 19:55:20 2021
 OS/Arch:           darwin/arm64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.8
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.16.6
  Git commit:       75249d8
  Built:            Fri Jul 30 19:53:34 2021
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.4.9
  GitCommit:        e25210fe30a0a703442421b0f60afac609f950a3
 runc:
  Version:          1.0.1
  GitCommit:        v1.0.1-0-g4144b63
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Output of docker info:

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Build with BuildKit (Docker Inc., v0.6.1-docker)
  compose: Docker Compose (Docker Inc., v2.0.0-rc.2)
  scan: Docker Scan (Docker Inc., v0.8.0)

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 3
 Server Version: 20.10.8
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: e25210fe30a0a703442421b0f60afac609f950a3
 runc version: v1.0.1-0-g4144b63
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 5.10.47-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 4
 Total Memory: 1.942GiB
 Name: docker-desktop
 ID: DNCI:7JKR:A2UM:CRJ5:SFVA:7GNU:6NKE:4CEC:I3IB:Y5U2:W4TD:W74C
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Additional environment details (AWS, VirtualBox, physical, etc.): MacBook Pro (13-inch, M1, 2020) macOS Big Sur 11.3.1 Processor Apple M1

Nimesh199398 avatar Jan 28 '22 08:01 Nimesh199398

Description

When building an image with a FROM local-image:latest statement with local-image being present (in my case offerplace-shared:latest), Docker anyway tries to pull that image from docker.io and fails. Since there seem to be any issues with image names that contain a dash, I also tried to replace that part with a repository name: offerplace/shared:latest, but to no avail - it still tries to pull it.

Steps to reproduce the issue:

  1. Build image local/image1
  2. Build image local/image2, with a FROM local/image1:latest statement
  3. Watch it failing

Describe the results you received:

Docker fails with the following error on my machine:

#1 [internal] load build definition from Dockerfile
#1 sha256:bb8fff504e2f268680b3eb7591394abfda1b2584b1e909984c56f0d067f89866
#1 transferring dockerfile: 511B done
#1 DONE 0.0s
#2 [internal] load .dockerignore
#2 sha256:f8cf604e196078a17ce397b978e6abb30bc09501f5d3094cfd91ee619b2e8483
#2 transferring context: 2B done
#2 DONE 0.0s
#3 [internal] load metadata for docker.io/library/openjdk:11-jre-slim
#3 sha256:0a99f1463278d5e63857a5e8637c92992bf2cd37d95a42501fa5494f1d624517
#3 CANCELED
#4 [internal] load metadata for docker.io/offerplace/shared:latest
#4 sha256:38ed34553b1dcac3fdf787bb8f99802fc2940aced9fbdcdf41bfab9b3f7f8bfd
#4 ERROR: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed
------
 > [internal] load metadata for docker.io/offerplace/shared:latest:
------
failed to solve with frontend dockerfile.v0: failed to create LLB definition: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

Describe the results you expected:

The build does not fail, since the image exists locally.

Additional information you deem important (e.g. issue happens only occasionally):

The issue arised suddenly and permanent without (at least as I remember) changing things. Even previously successfully built images cannot be built.

Output of docker version:

Client:
 Cloud integration: 1.0.17
 Version:           20.10.8
 API version:       1.41
 Go version:        go1.16.6
 Git commit:        3967b7d
 Built:             Fri Jul 30 19:55:20 2021
 OS/Arch:           darwin/arm64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.8
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.16.6
  Git commit:       75249d8
  Built:            Fri Jul 30 19:53:34 2021
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.4.9
  GitCommit:        e25210fe30a0a703442421b0f60afac609f950a3
 runc:
  Version:          1.0.1
  GitCommit:        v1.0.1-0-g4144b63
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Output of docker info:

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Build with BuildKit (Docker Inc., v0.6.1-docker)
  compose: Docker Compose (Docker Inc., v2.0.0-rc.2)
  scan: Docker Scan (Docker Inc., v0.8.0)

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 3
 Server Version: 20.10.8
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: e25210fe30a0a703442421b0f60afac609f950a3
 runc version: v1.0.1-0-g4144b63
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 5.10.47-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 4
 Total Memory: 1.942GiB
 Name: docker-desktop
 ID: DNCI:7JKR:A2UM:CRJ5:SFVA:7GNU:6NKE:4CEC:I3IB:Y5U2:W4TD:W74C
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Additional environment details (AWS, VirtualBox, physical, etc.): MacBook Pro (13-inch, M1, 2020) macOS Big Sur 11.3.1 Processor Apple M1

Experia

Nimesh199398 avatar Jan 28 '22 08:01 Nimesh199398

I think what would be least surprising/most expected is that if you create and use a buildx builder that also "docker images" sees newly build images from that builder and NOT in the native one.

Same goes for docker push, docker inspect and so on. They should ALL be made to use the currently active builds repository.

wederbrand avatar Mar 02 '22 13:03 wederbrand

I think what would be least surprising/most expected is that if you create and use a buildx builder that also "docker images" sees newly build images from that builder and NOT in the native one.

Same goes for docker push, docker inspect and so on. They should ALL be made to use the currently active builds repository.

I agree, its confusing that when using buildx on one machine like an m1 chip to set the platform to amd64 and then when building a new image on an actually amd64 platform the normal docker build and push to the image will cause the next pulls to ignore the latest.

j-mendez avatar Mar 06 '22 17:03 j-mendez

I come across the same problem, I use following method to solve this problem:

docker build --platform=linux/arm64 ....
docker run --platform=linux/arm64 ...

hitzhangjie avatar Jun 28 '22 08:06 hitzhangjie

For me, I had a buildx builder that was "corrupt" for lack of a better word. This worked for me...

# get the name of the multi-arch builder
docker buildx ls

docker buildx use default

docker buildx rm <multi-arch-builder-name>

Next time I created/used the multi-arch builder, it worked.

mikeoertli avatar Aug 16 '22 21:08 mikeoertli

Hi All, In my case the problem was about the platform settings

Adding --platform=linux/arm64 solved my problem in the short time

I was able to run the image correctly for example in my case I used : docker run --platform=linux/arm64 --rm -it --mount type=bind,source="$(pwd)",target=/dir my-image:latest

then I removed export DOCKER_DEFAULT_PLATFORM=linux/amd64/v8 from .bash_profile and this last step solved the problem

I hope this helps

linediconsine avatar Feb 22 '23 22:02 linediconsine

Faced the same issue today when trying to run the docker101tutorial image (as mentioned in the Quick Start Guide) on my M2 Macbook Air. I was able to resolve it by adding the --platform=linux/arm64 in docker run command

deyjishnu avatar Mar 19 '23 17:03 deyjishnu

Adding --platform=linux/arm64 to first and second image build commands worked for me. Should it be the default on Mac M2?

capripot avatar Sep 12 '23 07:09 capripot

Did you install ORB? I encountered the same problem: everytime I run "docker-compose up", it pulls the images from remote, even they exist in local. I did some search and no lucky.

Then I noticed the "docker-compose" and "docker" command are linked to orb; After uninstall ORB, and re-install docker desktop, the problem is gone.

shaofengshi avatar Mar 26 '24 09:03 shaofengshi