problem building `dev` on mac silicon ("pull access denied" or "insufficient_scope: authorization failed")

[crkrenn]

Expected behavior: docker compose pull or docker-compose up --build should work without much configuration.

Actual behavior:

% docker compose pull                
WARN[0000] The "GIT_HASH" variable is not set. Defaulting to a blank string. 
WARN[0000] The "GIT_HASH" variable is not set. Defaulting to a blank string. 
WARN[0000] The "GIT_HASH" variable is not set. Defaulting to a blank string. 
[+] Running 1/9
 ⠿ client-report Error                                                                                     1.4s
 ⠿ client-admin Error                                                                                      1.4s
 ⠿ nginx-proxy Error                                                                                       1.4s
 ⠿ file-server Error                                                                                       1.4s
 ⠿ server Error                                                                                            1.4s
 ⠿ postgres Error                                                                                          1.4s
 ⠿ math Error                                                                                              1.4s
 ⠿ client-participation Error                                                                              1.4s
 ⠿ maildev Pulled                                                                                          1.2s
WARNING: Some service image(s) must be built from source by running:
    docker compose build postgres server file-server client-participation math client-report client-admin
Error response from daemon: pull access denied for compdem/polis-postgres, repository does not exist or may require 'docker login': denied: requested access to the resource is denied

and

% docker-compose up --build
WARN[0000] The "GIT_HASH" variable is not set. Defaulting to a blank string. 
[+] Building 1.6s (38/73)                                                                                       
 => [compdem/polis-server:dev internal] load build definition from Dockerfile                              0.0s
 => => transferring dockerfile: 399B                                                                       0.0s
 => [compdem/polis-math:dev internal] load metadata for docker.io/library/clojure:tools-deps               0.9s
 => CANCELED [compdem/polis-file-server:dev internal] load metadata for docker.io/library/node:16.9.0-alp  0.5s
 => CANCELED [compdem/polis-file-server:dev internal] load metadata for docker.io/compdem/polis-client-ad  0.5s
 => CANCELED [compdem/polis-file-server:dev internal] load metadata for docker.io/compdem/polis-client-re  0.9s
 => ERROR [compdem/polis-file-server:dev internal] load metadata for docker.io/compdem/polis-client-parti  0.4s
 => [auth] compdem/polis-client-report:pull token for registry-1.docker.io                                 0.0s
 => [compdem/polis-server:dev internal] load build context                                                 0.1s
 => => transferring context: 1.10MB                                                                        0.0s
 => CANCELED [compdem/polis-server:dev 1/8] FROM docker.io/library/node:14.14.0-alpine@sha256:97c4cddbbf9  0.1s
 => CANCELED [compdem/polis-postgres:dev 1/2] FROM docker.io/library/postgres:13.4-alpine@sha256:c170270d  0.0s
 => CANCELED [compdem/polis-nginx-proxy:dev 1/4] FROM docker.io/library/nginx:1.21.3-alpine@sha256:1ff136  0.0s
 => CANCELED [compdem/polis-math:dev 1/4] FROM docker.io/library/clojure:tools-deps@sha256:9660c2723246a8  0.0s
 => CANCELED [compdem/polis-client-report:dev 1/8] FROM docker.io/library/node:11.15.0-alpine@sha256:8bb5  0.0s
 => CANCELED [compdem/polis-client-admin:dev internal] load build context                                  0.0s
 => CANCELED [compdem/polis-client-report:dev internal] load build context                                 0.0s
 => CANCELED [compdem/polis-client-participation:dev internal] load build context                          0.0s
 ------
 > [compdem/polis-file-server:dev internal] load metadata for docker.io/compdem/polis-client-participation:dev:
------
failed to solve: rpc error: code = Unknown desc = failed to solve with frontend dockerfile.v0: failed to create LLB definition: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

To Reproduce:

git clone https://github.com/compdemocracy/polis.git
docker login
docker compose pull

or

git clone https://github.com/compdemocracy/polis.git
docker login
docker-compose up --build

Screenshots: NA

Device information: OS X 11.6; Mac Silicon; Docker Desktop 4.2.0 (70708)

Additional context: Using default dev branch.

[crkrenn]

I googled the errors, but did not find much of use. I will try building on Digital Ocean later, but I would like to be able to develop on my laptop if possible. Thanks! -Chris

[andxblack]

I'm also having the same issue, but with Mac Intel Device information: OS X 12.1; Mac Intel; Docker Desktop 4.3.2 (72729)

[patcon]

@andxblack Ah, I think this is because the docker nightly builds haven't been set up since I transferred the dockerhub account back to @compdemocracy. So docker-compose pull commands won't work. Subscribe here for the eventual fix #1246

But the good news is that you can just use docker-compose up --build --detach. It will take a bit longer, since it's building the containers instead of pulling them from the internet, but it should work fine :)

I suspect your error is slightly different from @crkrenn's, so if the above suggestion doesn't work, please do share a copy of your logs :)

---

@crkrenn can you double-check from a fresh clone? I'm seeing the error you get is likely from some earlier advanced usage you did, maybe things lingering in .env related to docker buildkit: https://stackoverflow.com/a/66695181

[andxblack]

@patcon You're correct, it is a slightly different error. The above fix still doesn't seem to work though.

>> docker-compose up --build --detach
WARN[0000] The "GIT_HASH" variable is not set. Defaulting to a blank string.
WARN[0000] The "GIT_HASH" variable is not set. Defaulting to a blank string.
WARN[0000] The "GIT_HASH" variable is not set. Defaulting to a blank string.
[+] Building 3.5s (25/25) FINISHED
 => [compdem/polis-client-participation:dev internal] load build definiti  0.1s
 => => transferring dockerfile: 654B                                       0.0s
 => [compdem/polis-math:dev internal] load build definition from Dockerfi  0.0s
 => => transferring dockerfile: 169B                                       0.0s
 => [compdem/polis-postgres:dev internal] load build definition from Dock  0.0s
 => => transferring dockerfile: 362B                                       0.0s
 => [compdem/polis-client-admin:dev internal] load build definition from   0.0s
 => => transferring dockerfile: 355B                                       0.0s
 => [compdem/polis-nginx-proxy:dev internal] load build definition from n  0.0s
 => => transferring dockerfile: 38B                                        0.0s
 => [compdem/polis-file-server:dev internal] load build definition from D  0.0s
 => => transferring dockerfile: 612B                                       0.0s
 => [compdem/polis-client-report:dev internal] load build definition from  0.0s
 => => transferring dockerfile: 350B                                       0.0s
 => [compdem/polis-server:dev internal] load build definition from Docker  0.0s
 => => transferring dockerfile: 413B                                       0.0s
 => [compdem/polis-client-participation:dev internal] load .dockerignore   0.0s
 => => transferring context: 280B                                          0.0s
 => [compdem/polis-math:dev internal] load .dockerignore                   0.0s
 => => transferring context: 316B                                          0.0s
 => [compdem/polis-postgres:dev internal] load .dockerignore               0.0s
 => => transferring context: 105B                                          0.0s
 => [compdem/polis-client-admin:dev internal] load .dockerignore           0.0s
 => => transferring context: 510B                                          0.0s
 => [compdem/polis-nginx-proxy:dev internal] load .dockerignore            0.0s
 => => transferring context: 2B                                            0.0s
 => [compdem/polis-file-server:dev internal] load .dockerignore            0.1s
 => => transferring context: 2B                                            0.0s
 => [compdem/polis-client-report:dev internal] load .dockerignore          0.1s
 => => transferring context: 196B                                          0.0s
 => [compdem/polis-server:dev internal] load .dockerignore                 0.1s
 => => transferring context: 34B                                           0.0s
 => CANCELED [compdem/polis-client-admin:dev internal] load metadata for   2.6s
 => CANCELED [compdem/polis-math:dev internal] load metadata for docker.i  2.7s
 => CANCELED [compdem/polis-postgres:dev internal] load metadata for dock  2.7s
 => CANCELED [compdem/polis-nginx-proxy:dev internal] load metadata for d  2.6s
 => CANCELED [compdem/polis-file-server:dev internal] load metadata for d  2.5s
 => CANCELED [compdem/polis-file-server:dev internal] load metadata for d  2.5s
 => CANCELED [compdem/polis-file-server:dev internal] load metadata for d  2.5s
 => ERROR [compdem/polis-file-server:dev internal] load metadata for dock  2.5s
 => CANCELED [compdem/polis-server:dev internal] load metadata for docker  2.5s
------
 > [compdem/polis-file-server:dev internal] load metadata for docker.io/compdem/polis-client-report:dev:
------
failed to solve: rpc error: code = Unknown desc = failed to solve with frontend dockerfile.v0: failed to create LLB definition: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

[patcon]

Oooooooh. Shit ok, I totally get what this is. It's an issue with building file-server, because it's dockerfile extracts files from existing container images. And the first time, and those images for client-admin, client-participation and client-report don't yet exist..! (@crkrenn, this is that weird thing I was talking about in https://github.com/compdemocracy/polis/pull/1277#issuecomment-1011145511)

Running this should solve it:

docker-compose build client-admin client-participation client-report 
docker-compose up --build --detach

We should probably add a Makefile entry for make docker-up to run this easily, as everyone should be using this instead of any other use of docker-compose up. (Otherwise, docker-compose commands need to be run twice to be guaranteed that the most recent code is in the container image.)

[patcon]

@andxblack Lemme know if that works :) And if it does, if you feel like making a first contribution: a pull request to fix up the README and remove mention of docker-compose pull would be supremely appreciated. But no pressure!

[andxblack]

Thanks @patcon that works. Will fix up the readme tomorrow.

[crkrenn]

Woo Hoo!

docker-compose build client-admin client-participation client-report 
docker-compose up --build --detach

Worked.

[crkrenn]

I think this can be closed. I've had no trouble building on mac silicon lately.