big-ann-benchmarks icon indicating copy to clipboard operation
big-ann-benchmarks copied to clipboard

Error on Apple M1 chip

Open LysanderT opened this issue 1 year ago • 3 comments

when running python install.py --neurips23track filter --algorithm faiss and after waiting for 40 minutes, it finally output:

What's Next?
  View summary of image vulnerabilities and recommendations → docker scout quickview
Building algorithm images... with (1) processes
Building neurips23-filter-faiss...
docker build  --rm -t neurips23-filter-faiss -f neurips23/filter/faiss/Dockerfile .
[+] Building 1413.5s (8/15)                                                                                                                                  docker:desktop-linux
 => [internal] load build definition from Dockerfile                                                                                                                         0.0s
 => => transferring dockerfile: 990B                                                                                                                                         0.0s
 => [internal] load .dockerignore                                                                                                                                            0.0s
 => => transferring context: 103B                                                                                                                                            0.0s
 => [internal] load metadata for docker.io/library/neurips23:latest                                                                                                          0.0s
 => [ 1/11] FROM docker.io/library/neurips23                                                                                                                                 0.0s
 => [internal] load build context                                                                                                                                            0.0s
 => => transferring context: 4.86kB                                                                                                                                          0.0s
 => [ 2/11] RUN apt update && apt install -y wget swig                                                                                                                      22.7s
 => [ 3/11] RUN wget https://repo.anaconda.com/archive/Anaconda3-2023.03-0-Linux-x86_64.sh                                                                                1388.3s
 => ERROR [ 4/11] RUN bash Anaconda3-2023.03-0-Linux-x86_64.sh -b                                                                                                            2.3s
------
 > [ 4/11] RUN bash Anaconda3-2023.03-0-Linux-x86_64.sh -b:
0.163 PREFIX=/root/anaconda3
2.243 Unpacking payload ...
2.246 qemu-x86_64: Could not open '/lib64/ld-linux-x86-64.so.2': No such file or directory
------
Dockerfile:5
--------------------
   3 |     RUN apt update && apt install -y wget swig
   4 |     RUN wget https://repo.anaconda.com/archive/Anaconda3-2023.03-0-Linux-x86_64.sh
   5 | >>> RUN bash Anaconda3-2023.03-0-Linux-x86_64.sh -b
   6 |
   7 |     ENV PATH /root/anaconda3/bin:$PATH
--------------------
ERROR: failed to solve: process "/bin/sh -c bash Anaconda3-2023.03-0-Linux-x86_64.sh -b" did not complete successfully: exit code: 255


Install Status:
{'neurips23-filter-faiss': 'fail'}

LysanderT avatar Sep 16 '23 06:09 LysanderT

Could you try the suggested solutions here: https://stackoverflow.com/questions/71040681/qemu-x86-64-could-not-open-lib64-ld-linux-x86-64-so-2-no-such-file-or-direc? In particular, try adding --platform=linux/amd64 to our base image.

I'm not sure whether the whole setup will make much sense from a performance point of view, since you will emulate x86 on your ARM-based CPU, but for testing it should probably work.

maumueller avatar Sep 27 '23 06:09 maumueller

@LysanderT Confirming that the issue exists on my M1 laptop. If what @maumueller suggested does not work, please let us know and we will try to come up with a way to run things locally on an M1 laptop for this track. Also, I would suggest testing on a linux VM (and an intel CPU).

ingberam avatar Oct 09 '23 13:10 ingberam

Hey, just to say that at least for the sparse track --platform=linux/amd64 made it work (even if it is kinda slow). I can make a pull request later showing how I did it.

Also, I expect to send a pull request soon (hopefully tomorrow?) with our entry for the sparse track with 3 measurements to try to give you an idea of the differences (azure, my M1 and our best CPU server)

cadurosar avatar Oct 11 '23 16:10 cadurosar