Failed to solve with frontend dockerfile.v0: failed to create LLB definition: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

[gintsmurans]

Hi!

Description of the issue

I recently updated docker and after update it seems that Dockerfile cannot use local images anymore. I have this instance:

Which is later used by another service in Dockerfile using FROM services_develop:staging. Prior update it was working just fine, now only option seems to be to create a local registry. Is this really intended?

Context information (for bug reports)

Output of docker-compose version

docker-compose version 1.29.2, build 5becea4c
docker-py version: 5.0.0
CPython version: 3.9.0
OpenSSL version: OpenSSL 1.1.1h  22 Sep 2020

/!\ If docker-compose version reports v2.0...` you're using the wrong repository, please report issues on Compose-CLI

Output of docker version

Client:
 Cloud integration: 1.0.17
 Version:           20.10.7
 API version:       1.41
 Go version:        go1.16.4
 Git commit:        f0df350
 Built:             Wed Jun  2 11:56:23 2021
 OS/Arch:           darwin/arm64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.7
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       b0f5bc3
  Built:            Wed Jun  2 11:55:36 2021
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.4.6
  GitCommit:        d71fcd7d8303cbf684402823e425e9dd2e99285d
 runc:
  Version:          1.0.0-rc95
  GitCommit:        b9ee9c6314599f1b4a7f497e1f1f856fe433d3b7
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Output of docker-compose config (Make sure to add the relevant -f and other flags)

networks:
  default:
    driver: bridge
  pm_default:
    external: true
    name: pm_default
services:
  build:
    build:
      args:
        SRC_MOUNT_PATH: /srv/sites/services
      context: /Users/xxx/Documents/Projects/sites/yyy/services/docker/build
    depends_on:
      develop:
        condition: service_started
    environment:
      DEPLOYMENT_PROD_HOSTS: ''
      DEPLOYMENT_PROD_KEY_FILENAME: ''
      DEPLOYMENT_PROD_KEY_PASSPHRASE: ''
      DEPLOYMENT_PROD_LOCAL_SCRIPTS_PATH: ''
      DEPLOYMENT_PROD_REMOTE_SCRIPTS_PATH: ''
      DEPLOYMENT_PROD_RUNTIME_PATH: /srv/sites/yyy/services
      DEPLOYMENT_PROD_TEST_RUNTIME_PATH: /srv/sites/yyy/testing.services
      DEPLOYMENT_PROD_TMP_PATH: /tmp
      DEPLOYMENT_PROD_USERNAME: root
      RUNTIME_PATH: /srv/sites/services/pm_services
      SRC_MOUNT_PATH: /srv/sites/services
    image: services_build:staging
    networks:
      default: null
    volumes:
    - /Users/xxx/Documents/Projects/sites/yyy/services:/srv/sites/services:cached
  develop:
    build:
      args:
        SRC_MOUNT_PATH: /srv/sites/services
      context: /Users/xxx/Documents/Projects/sites/yyy/services/docker/develop
    depends_on:
      redisdb:
        condition: service_started
    environment:
      DEPLOYMENT_PROD_HOSTS: ''
      DEPLOYMENT_PROD_KEY_FILENAME: ''
      DEPLOYMENT_PROD_KEY_PASSPHRASE: ''
      DEPLOYMENT_PROD_LOCAL_SCRIPTS_PATH: ''
      DEPLOYMENT_PROD_REMOTE_SCRIPTS_PATH: ''
      DEPLOYMENT_PROD_RUNTIME_PATH: /srv/sites/yyy/services
      DEPLOYMENT_PROD_TEST_RUNTIME_PATH: /srv/sites/yyy/testing.services
      DEPLOYMENT_PROD_TMP_PATH: /tmp
      DEPLOYMENT_PROD_USERNAME: root
      RUNTIME_PATH: /srv/sites/services/pm_services
      SRC_MOUNT_PATH: /srv/sites/services
    image: services_develop:staging
    networks:
      default: null
      pm_default: null
    ports:
    - published: 4100
      target: 4100
    - published: 4200
      target: 4110
    volumes:
    - /Users/xxx/Documents/Projects/sites/yyy/services:/srv/sites/services:cached
  redisdb:
    image: redis:alpine
version: '3.9'

Steps to reproduce the issue

1. docker-compose up --build build

Observed result

Fails to run

Expected result

Container is run

Stacktrace / full error message

#2 [services_build:staging internal] load build definition from Dockerfile
#2 sha256:904e5663af5547b640ab36250d9303488863d83cb6ca4c0111b17700a843eec9
#2 transferring dockerfile: 337B done
#2 DONE 0.0s

#1 [services_develop:staging internal] load build definition from Dockerfile
#1 sha256:d8973824e1cc852c8a3ba858f1c282f2943a049f4c86b2cc88e4ecfb20912cb3
#1 transferring dockerfile: 860B done
#1 DONE 0.0s

#3 [services_develop:staging internal] load .dockerignore
#3 sha256:a577a2d1f67337b83d0055cf11cf3d65ec16157e002120d74886257da7875479
#3 transferring context: 2B done
#3 DONE 0.0s

#4 [services_develop:staging internal] load metadata for docker.io/library/debian:stable
#4 sha256:03abc9f10f935e7137bbfe987d86a0879bf4bcb2ca500f3919314c6aa058f2f8
#4 ...

#5 [services_build:staging internal] load .dockerignore
#5 sha256:a3a79252ae83d732c535c559ffb437f9be6ae841462892d5b0c084a22f5ebf2b
#5 transferring context: 2B done
#5 DONE 0.0s

#6 [services_build:staging internal] load metadata for docker.io/library/services_develop:staging
#6 sha256:0aabca4baec97ad783f7b602174718f61033d1eb5568a3e962ea3e70e39858bb
#6 ERROR: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

#4 [services_develop:staging internal] load metadata for docker.io/library/debian:stable
#4 sha256:03abc9f10f935e7137bbfe987d86a0879bf4bcb2ca500f3919314c6aa058f2f8
#4 CANCELED
------
 > [services_build:staging internal] load metadata for docker.io/library/services_develop:staging:
------
failed to solve: rpc error: code = Unknown desc = failed to solve with frontend dockerfile.v0: failed to create LLB definition: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

Additional information

Macbook Pro M1, MacOS 11.4.

[MartinFalatic]

I'm using Docker Desktop 3.5.2 on macOS 11.5.1 (Intel hardware).

I just ran into this today (recently cleared out my local images so I was rebuilding from scratch). It kept looking on docker.io for what were clearly local images I was in the process of trying to build, then failing with:

...
------
 > [my_internal_app_name_tests internal] load metadata for docker.io/my_internal_app_name/app:latest:
------
------
 > [my_internal_app_name/app internal] load build context:
------
failed to solve: rpc error: code = Unknown desc = failed to solve with frontend dockerfile.v0: failed to create LLB definition: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

Under Experimental Features "Use Docker Compose V2" was selected. I deselected that, restarted Docker, and after that docker-compose worked properly as it always has. (Apparently docker-compose disable-v2 will have the same effect, but you probably still should restart Docker afterward.)

The man page at https://docs.docker.com/compose/cli-command/ states that V2 is in beta and that they will gradually turn this option on automatically for Docker Desktop users. I don't recall opting into that myself.

I think this has been set for a while now but I didn't notice this sooner because I'd already built my app at least once before and that local image was already present - til I wiped everything today, that is.

[gintsmurans]

Looks like I already had V2 disabled, then I tried to use image when enabled then again when disabled - nothing changes:

#3 [internal] load metadata for docker.io/library/external_services_develop:latest
#3 sha256:796a096830b8eb3bfae3531721a1dee705af9d239f65cd75ce0bee93aa3490f9
#3 ERROR: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed
------
 > [internal] load metadata for docker.io/library/external_services_develop:latest:
------
failed to solve with frontend dockerfile.v0: failed to create LLB definition: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed
Service 'build' failed to build : Build failed

It still tries to load local image from docker.io.

[gintsmurans]

This seems to be related to me having M1 Macbook. I am running same build command on windows WSL just fine.

[gintsmurans]

After reading comments from @ndeloof on the #8538 here is what I have found:

1. Firstly turns out depends_on are parallel, thats why you first need to build all dependencies and only then can use built images. Correct me if I am wrong.
2. docker-compose and docker compose both works on intel macbook and also on windows wsl.
3. docker compose build doesn't work on M1 macbook.
4. docker-compose build works on M1 macbook only when DOCKER_BUILDKIT environment variable is set to 0
5. docker-compose up does not work on M1 even with DOCKER_BUILDKIT set to 0, so you first need to build image and then "up it" without --build flags.

[ndeloof]

This is indeed a buildkit limitation. cc @tonistiigi

[jonycodes]

I'm also hitting this on windows all the sudden, docker desktop 4.0.1, engine 20.10.8 compose 1.29.2. Is there a work around?

[MartinFalatic]

I'm also hitting this on windows all the sudden, docker desktop 4.0.1, engine 20.10.8 compose 1.29.2. Is there a work around?

Hit me too on a new machine even though I know I didn't opt in to their broken docker-compose 2.0 beta. Sure enough, I'd been opted in at some point without my consent. Unchecked the option in the dashboard and it worked fine.

[ndeloof]

This issue depends on https://github.com/docker/buildx/issues/447

applies both on Docker Compose V2 and V1 as long as Buildkit is enabled (which is set by default on Docker Desktop)

[GF-Huang]

1>#1 [internal] load build definition from Dockerfile
1>#1 sha256:956bc10a1f2e1ceb1467f46c35bd8f35881c8bce89860af10d947bf29232184a
1>#1 transferring dockerfile: 32B 0.0s done
1>#1 DONE 0.0s
1>#2 [internal] load .dockerignore
1>#2 sha256:39e216f61cef12c49ac62239816a4769fd6ed8537fb3a48af81544abad0525d7
1>#2 transferring context: 34B done
1>#2 DONE 0.0s
1>#3 [internal] load metadata for docker.io/microsoft/dotnet:2.2-aspnetcore-runtime
1>#3 sha256:2209054a124f7bc1424fa6735d588f36916bcec272f45a5e3e54f7288e65d73e
1>#3 ...
1>#4 [auth] microsoft/dotnet:pull token for registry-1.docker.io
1>#4 sha256:209aca23d26d7a46457af8b422359c9c5f58113ed548183047a1fee47c3b5738
1>#4 DONE 0.0s
1>#3 [internal] load metadata for docker.io/microsoft/dotnet:2.2-aspnetcore-runtime
1>#3 sha256:2209054a124f7bc1424fa6735d588f36916bcec272f45a5e3e54f7288e65d73e
1>#3 ERROR: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

[PeterLaComb-Availity]

I ran in to this issue this morning (setting up a new machine). Disabling the Docker Engine "buildkit" feature works for now:

"features": { "buildkit": false }

[Karanjot-singh]

I ran in to this issue this morning (setting up a new machine). Disabling the Docker Engine "buildkit" feature works for now:

"features": { "buildkit": false }

I can confirm this works for MacBook air M1. Thanks!

[Microhive]

"features": { "buildkit": false }

Can also confirm this works on my Macbook Pro M1.

[gavin-shaw]

This worked for me too thank you.

In case anyone is confused like I was briefly, there is a "features" config node in both devcontainer.json and your local Docker configuration.

You must change your local Docker configuration, not devcontainer.json:

[wishma2013]

"features": { "buildkit": false } Can also confirm this works on my Macbook Pro of macOS Big Sur 11.5.2.

[JoshuaSchlichting]

"features": { "buildkit": false } appears to cause my Docker daemon to hang as warned.

MacBook Pro (14-inch, 2021) with Apple M1 Pro chip macOS Monterey Version 12.2

[MarcStoecker]

50/50 for the "features": { "buildkit": false } workaround for me:

• works for Win/x64 on my main machine
• not working for my Mac/M1 (daemon hangs)

[Artem-Haholkin-deepsee]

For me worked on m1 chip add arch FROM --platform=linux/amd64 ubunutu:16.04

[alex-oswald]

Worked for me on MacBook Pro with M1 pro chip. Thanks.

[vosechu]

Ahhhhhhh, @Artem-Haholkin-deepsee got me working. I built a local image using --platform=linux/amd64, but then forgot to put that --platform on the next image in the chain. So, since I'm on an M1, it tried to find an arm variant (which I hadn't built), and failed. Presumably, it then tried to check in with dockerhub, and for whatever reason got a 403.

[521xueweihan]

with CLI also can use：export DOCKER_BUILDKIT=0 to close buildkit.

[D1no]

Just had the same issue when setting up visual studio code remote dev container on my M1 mac — weird as I believe it worked before. Docker Desktop v4.9.0

"features": { "buildkit": false } Changed as seen below in the docker desktop app, and works now.

[jjpinto]

Windows 10 Enterprise Docker desktop -> Settings -> Docker engine (v20.10.16) "features": { "buildkit": false } -

Docker hangs - had to uninstall it.

[rajeshatverbtech]

None of these worked for me on MacBook Pro 2021 with M1 Max The daemon does not hang, but the issue persists.

[seadub]

Tried setting to false to no avail.

System: MacBook Pro18,1 Apple M1 Pro Docker Engine v20.10.17

Config:

{
  "builder": {
    "gc": {
      "defaultKeepStorage": "20GB",
      "enabled": true
    }
  },
  "debug": false,
  "experimental": false,
  "features": {
    "buildkit": false
  },
  "insecure-registries": [
    "..."
  ],
  "registry-mirrors": []
}

[umama931]

Also a M1 user, None of the solution above helped me either, still getting failed to solve with frontend dockerfile.v0: failed to create LLB definition: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed is this a docker desktop issue?

[perymerdeka]

Also a M1 user, None of the solution above helped me either, still getting failed to solve with frontend dockerfile.v0: failed to create LLB definition: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed is this a docker desktop issue?

this is a new issue of docker desktop, on M1, I try to build it on my windows machine is work well

[abegascia]

A solution for MacOS Monterrey 12.3.1

Set buildkit to false in Docker Desktop app

And run in terminal this commands:

• export DOCKER_BUILDKIT=0
• export COMPOSE_DOCKER_CLI_BUILD=0

It works fine to me.

[PatelTheKanu]

Disabling buildkit as shown above/below worked for me:

"features": {
    "buildkit": false
  }

System: Macbook pro 2021 M1 Max Monterey 12.6

[enyakucera]

It helped me to solve same problem on Docker v20.10.21 on ThinkPad/Win10 :-)

[leoliuowi]

it also happned to me. i used "docker-compose" instead of "docker compose" to walk around this issue. this error only happens if I am building for the first time, if i am doing a rebuild, it won't happen. I believe this is a bug