Problem Building using docker-compose

[Krutarth-Bhatt]

I am trying to build EvalAI on my Macbook Air M1 and getting the below error at the end.

[Ram81]

Hi @Krutarth-Bhatt,

We don't officially support EvalAI development environment for Macbook M1. Some contributors have found out a fix for this issue by adding this change in nodejs dockerfile

FROM --platform=linux/amd64 node:12.4.0

If this doesn't work you can post the issue on our gitter channel where someone can help you resolve this issue.

[Krutarth-Bhatt]

Can you please tell me in which while I need to add this??

[AnkitGupta17134]

Hi @Krutarth-Bhatt

Please add above line to two files present at:

1. docker/dev/nodejs/Dockerfile
2. docker/dev/nodejs_v2/Dockerfile

If this doesn't work you can post the issue on our gitter channel where someone can help you resolve this issue.

[Krutarth-Bhatt]

Hey, Thanks a lot for the response. I am getting this message while I am trying to compile after the change that you suggested. Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

[AnkitGupta17134]

Hey,

If docker isn't running Check if docker is running or not using: sudo service docker status If not running, run this: sudo service docker start

If permission of docker unix socket is problem Check about permission of socket: sudo ls -la /var/run/docker.sock Grant the necessary ownership using : sudo chown [username]:docker /var/run/docker.sock

[Krutarth-Bhatt]

Sorry I forgot to turn on the docker. No I am getting this error: failed to solve: rpc error: code = Unknown desc = failed to solve with frontend dockerfile.v0: failed to create LLB definition: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

[gchhablani]

Hi @Ram81 I am facing the docker error shown in the first image as well.

 > [evalai_worker internal] load metadata for docker.io/library/evalai_django:latest:
------
failed to solve: rpc error: code = Unknown desc = failed to solve with frontend dockerfile.v0: failed to create LLB definition: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

[nantonzhang]

Hi @gchhablani

I had the issue before and found this link (https://stackoverflow.com/questions/64221861/an-error-failed-to-solve-with-frontend-dockerfile-v0/66695181#66695181) works for me.

[FlinkRay]

Hi @Ram81 I am facing the docker error shown in the first image as well.

 > [evalai_worker internal] load metadata for docker.io/library/evalai_django:latest:
------
failed to solve: rpc error: code = Unknown desc = failed to solve with frontend dockerfile.v0: failed to create LLB definition: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

Hi,I am also encountering this issue.Do you sloved this issue?

[FrancescoSaverioZuppichini]

Same issue on linux

[gchhablani]

@FrancescoSaverioZuppichini Did this fix not work for you?

[FrancescoSaverioZuppichini]

Attaching some output from DOCKER_BUILDKIT=0 docker-compose up --build

...
Removing intermediate container 38088e09b835
 ---> 2b7f4ec3b136
Step 10/11 : ADD . /code
 ---> 2275cd802cda
Step 11/11 : CMD ["./docker/wait-for-it.sh", "django:8000", "--", "python", "-m", "scripts.workers.submission_worker"]
 ---> Running in f7e75dec37de
Removing intermediate container f7e75dec37de
 ---> 531320dc5549
Successfully built 531320dc5549
Successfully tagged evalai-worker_py3_8:latest
Sending build context to Docker daemon  91.32MB
Step 1/20 : FROM node:14.20.0
14.20.0: Pulling from library/node
86467c57892b: Pull complete 
2b34aaa3cba7: Pull complete 
5a443ab54eaa: Pull complete 
e6cacf42b567: Pull complete 
2334d631be5e: Pull complete 
9707f1b650a1: Pull complete 
0940c720eb18: Pull complete 
7ecc5f2d0437: Pull complete 
3b02042b8ccb: Pull complete 
Digest: sha256:6adfb0c2a9db12a06893974bb140493a7482e2b3df59c058590594ceecd0c99b
Status: Downloaded newer image for node:14.20.0
 ---> 4f585a7a4342
Step 2/20 : RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add -
 ---> Running in 2d5a4895c907
Warning: apt-key output should not be parsed (stdout is not a terminal)
OK
Removing intermediate container 2d5a4895c907
 ---> 525727702f34
Step 3/20 : RUN sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list'
 ---> Running in 676dd40fe4c9
Removing intermediate container 676dd40fe4c9
 ---> d02b988e76e3
Step 4/20 : RUN apt-get update && apt-get install -yq google-chrome-stable
 ---> Running in 8e318c6446a8
Get:1 http://deb.debian.org/debian buster InRelease [122 kB]
Get:2 http://dl.google.com/linux/chrome/deb stable InRelease [1811 B]
Get:3 http://deb.debian.org/debian-security buster/updates InRelease [34.8 kB]
Get:4 http://deb.debian.org/debian buster-updates InRelease [56.6 kB]
Err:1 http://deb.debian.org/debian buster InRelease
  At least one invalid signature was encountered.
Err:2 http://dl.google.com/linux/chrome/deb stable InRelease
  At least one invalid signature was encountered.
Err:3 http://deb.debian.org/debian-security buster/updates InRelease
  At least one invalid signature was encountered.
Err:4 http://deb.debian.org/debian buster-updates InRelease
  At least one invalid signature was encountered.
Reading package lists...
W: GPG error: http://deb.debian.org/debian buster InRelease: At least one invalid signature was encountered.
E: The repository 'http://deb.debian.org/debian buster InRelease' is not signed.
W: GPG error: http://dl.google.com/linux/chrome/deb stable InRelease: At least one invalid signature was encountered.
E: The repository 'http://dl.google.com/linux/chrome/deb stable InRelease' is not signed.
W: GPG error: http://deb.debian.org/debian-security buster/updates InRelease: At least one invalid signature was encountered.
E: The repository 'http://deb.debian.org/debian-security buster/updates InRelease' is not signed.
W: GPG error: http://deb.debian.org/debian buster-updates InRelease: At least one invalid signature was encountered.
E: The repository 'http://deb.debian.org/debian buster-updates InRelease' is not signed.
2 errors occurred:
        * Status: pull access denied for evalai_django, repository does not exist or may require 'docker login': denied: requested access to the resource is denied, Code: 1
        * Status: The command '/bin/sh -c apt-get update && apt-get install -yq google-chrome-stable' returned a non-zero code: 100, Code: 100
        ```

[FrancescoSaverioZuppichini]

@FrancescoSaverioZuppichini Did this fix not work for you?

Doesn't seem to fix the issue, not sure but I am on Ubuntu 22.04

Seems to be a permission issue for evalai_django

[FrancescoSaverioZuppichini]

building the containers works individually

docker build -t evalai_nodejs -f docker/dev/nodejs/Dockerfile . -> worked docker build -t evalai_django -f docker/dev/django/Dockerfile . -> worked docker build -t evalai_worker -f docker/dev/worker/Dockerfile . -> worked .... same for py3.8 -> worked

So I rebuild everything with docker-compose and everything is fine now - no idea :face_with_head_bandage:

[FrancescoSaverioZuppichini]

Fresh installation on fresh Ubuntu machine, by running DOCKER_BUILDKIT=0 docker-compose up --build

Removing intermediate container b6a7f5dee812
 ---> fba7977f2215
Step 19/24 : CMD ["gulp", "dev:runserver"]
 ---> Running in bfa063bb2665
Removing intermediate container bfa063bb2665
 ---> cd0cc1598748
Step 20/24 : EXPOSE 8888
 ---> Running in c25e658ce9cf
Removing intermediate container c25e658ce9cf
 ---> fb7447f8523f
Step 21/24 : LABEL com.docker.compose.image.builder=classic
 ---> Running in c39ac31714df
Removing intermediate container c39ac31714df
 ---> 9fca23486383
Step 22/24 : LABEL com.docker.compose.project=evalai
 ---> Running in abb0367d95d7
Removing intermediate container abb0367d95d7
 ---> 95ccf1446a16
Step 23/24 : LABEL com.docker.compose.service=nodejs
 ---> Running in c196735007b2
Removing intermediate container c196735007b2
 ---> c363ab9d918b
Step 24/24 : LABEL com.docker.compose.version=2.16.0
 ---> Running in abfa201863a9
Removing intermediate container abfa201863a9
 ---> af91b55c3fa1
Successfully built af91b55c3fa1
Successfully tagged evalai-nodejs:latest
1 error occurred:
        * Status: pull access denied for evalai_django, repository does not exist or may require 'docker login': denied: requested access to the resource is denied, Code: 1

[zhimin-z]

I am also having exactly the same issue building the project on Macbook Air M2

[shubhsboson]

This thread here sheds more light on the problem, seems this is docker/buildkit related and not specific to OS. It also lists multiple solutions. Unfortunately some work for some.

For my Mac M1 Air

I disabled docker compose v2 from docker dashboard

And then quit docker desktop manually and started it again to be sure that settings are updated. And things worked not completely though. I still see issues while installing google-chrome-stable

and @Ram81 's fix

FROM --platform=linux/amd64 node:12.4.0

Throws the following error

Here's what solved this issue

This was added in docker/dev/nodejs/Dockerfile

But all's not well yet, I am getting the following error which seems to be coming from a wrong requirement spec for matlplotlib as v3.3.3 shouldn't throw this error

Okay my bad the issue was right in from of me, it's not coming from matplotlib but numpy-1.24.2 Not sure how it gets here as couldn't find any requirements.txt referring to this.

Okay following worked for me

This was added in docker/dev/worker/Dockerfile

I now have the dev stage evalai running on my machine

Here's a summary of what worked for me:

1. Disabling Use Docker Compose V2 from docker dashboard and manually quitting and starting docker again.
2. Replacing line 1 in docker/dev/nodejs/Dockerfile with

FROM --platform=linux/amd64 node:14.20.0

3. And replacing line 7 in docker/dev/worker/Dockerfile with

RUN pip install -U cffi service_identity cython==0.29 numpy==1.21.0

Details of my setup/config: Branch: master @ https://github.com/Cloud-CV/EvalAI/commit/fbfc7f04cf8570704fd41808c0c5f5cfed518505 Machine: Macbook Air M1 2020 OS Version: Ventura 13.2.1 (22D68) Stage: dev