seung-lab
Installation in a Docker container fails
I'm trying to install CloudVolume, but DracoPy is a dependency of CloudVolume and it's DracoPy that's giving me the following error on installation with pip:
File "/home/node/.local/lib/python3.11/site-packages/skbuild/setuptools_wrap.py", line 453, in setup
297.6 _check_skbuild_parameters(cmake_install_dir, cmake_source_dir)
297.6 File "/home/node/.local/lib/python3.11/site-packages/skbuild/setuptools_wrap.py", line 282, in _check_skbuild_parameters
297.6 raise SKBuildError(msg)
297.6
297.6
297.6 setup parameter 'cmake_source_dir' set to a nonexistent directory.
297.6 Project Root : /tmp/pip-install-81onv6ad/dracopy_b9dd79e7ca6f4185a0ad90e446d45051
297.6 CMake Source Directory: ./draco
Here's the relevant parts of my Dockerfile:
FROM node-base as python-base
RUN apk add python3 python3-dev py3-pip
FROM python-base as with-build-dependencies
RUN apk add build-base
RUN apk add cmake
RUN apk add zlib-dev
FROM with-build-dependencies as python-requirements
USER node
RUN pip install numpy --break-system-packages
RUN pip install Cython --break-system-packages
RUN pip install scikit-build --break-system-packages
RUN pip install pandas --break-system-packages
RUN pip install -r requirements.txt --break-system-packages
Generally, whenever someone tries to install from source, they're in for a bad time.
I tried running the above docker file and got this message on MacOS:
failed to solve with frontend dockerfile.v0: failed to create LLB definition: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed
Is there a reason your installation is not finding one of these wheels?
https://pypi.org/project/DracoPy/#files
CloudVolume's in the requirements file, I didn't think I was installing from source. I'm relatively new to Docker and trying to figure out what the heck to do.
Here's the full Dockerfile, for testing:
FROM node:lts-alpine as node-base
RUN mkdir -p /home/node/app/node_modules && chown -R node:node /home/node/app
WORKDIR /home/node/app
COPY --chown=node:node package*.json ./
COPY --chown=node:node requirements.txt ./
RUN apk update
RUN apk upgrade
FROM node-base as python-base
RUN apk add python3 python3-dev py3-pip
FROM python-base as with-build-dependencies
RUN apk add build-base
RUN apk add cmake
RUN apk add zlib-dev
FROM with-build-dependencies as python-requirements
USER node
RUN pip install numpy --break-system-packages
RUN pip install Cython --break-system-packages
RUN pip install scikit-build --break-system-packages
RUN pip install pandas --break-system-packages
RUN pip install -r requirements.txt --break-system-packages
FROM python-requirements
USER node
RUN npm install
COPY --chown=node:node . .
EXPOSE 443
CMD node app.js
And here's requirements.txt.
nibabel
indexed-gzip
numpy
argparse
imageio
cloud-volume
Pillow
#For tiffs
tifffile
#DICOM ZIP archives
pydicom
#XLSX sheets (Spreadsheet for Neuroglancer Segmentation configuration)
pandas
#Pandas requires openpyxl to load xlsx sheets.
openpyxl
matplotlib
Hey, I'm running in the same issue:
23.34 Collecting DracoPy==1.4.0 (from -r /app/requirements.txt (line 34))
23.34 Downloading DracoPy-1.4.0.tar.gz (158 kB)
23.34 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 159.0/159.0 kB 14.1 MB/s eta 0:00:00
23.34 Preparing metadata (setup.py): started
23.34 Preparing metadata (setup.py): finished with status 'error'
23.34 error: subprocess-exited-with-error
23.34
23.34 × python setup.py egg_info did not run successfully.
23.34 │ exit code: 1
23.34 ╰─> [11 lines of output]
23.34 Traceback (most recent call last):
23.34 File "/usr/local/lib/python3.11/dist-packages/skbuild/setuptools_wrap.py", line 453, in setup
23.34 _check_skbuild_parameters(cmake_install_dir, cmake_source_dir)
23.34 File "/usr/local/lib/python3.11/dist-packages/skbuild/setuptools_wrap.py", line 282, in _check_skbuild_parameters
23.34 raise SKBuildError(msg)
23.34
23.34
23.34 setup parameter 'cmake_source_dir' set to a nonexistent directory.
23.34 Project Root : /tmp/pip-install-8y7lihlm/dracopy_e2b7517195ea4944b274224bc1f1cbec
23.34 CMake Source Directory: ./draco
23.34
23.34 [end of output]
23.34
23.34 note: This error originates from a subprocess, and is likely not a problem with pip.
23.34 error: metadata-generation-failed
23.34
23.34 × Encountered error while generating package metadata.
23.34 ╰─> See above for output.
23.34
23.34 note: This is an issue with the package mentioned above, not pip.
23.34 hint: See above for details.
I'm running an Ubuntu 20.04 image on macOS 14.3. I have an M1 chip. I don't believe there is a manylinux build for arm64 so that's probably the issue
Interesting... it seems I do have aarch64 in the build commands, but I also have musllinux skipped because it takes forever to build. Do you think unskipping musllinux would solve your problem?
That's weird because I don't seem to find any aarch64 binaries here: https://pypi.org/project/DracoPy/1.4.0/#files
Also it's not really an issue for me since I can also build my docker with the --platform linux/amd64 flag and then it uses rosetta to use x86_64 instead of arm64
I looked into this a bit more and it's odd, I don't know why those wheels didn't get built. Maybe after I bump the cibuildwheel it's produce them. My other repo that has a very similar Github Action configuration built them. I should give that a try.
It looks like the issue is I'm cross compiling for arm64 and Cmake needs an ARM OpenSSL dependency installed.
Hi, are there any updates about this issue?
Downloading DracoPy-1.4.0.tar.gz (158 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 159.0/159.0 kB 4.5 MB/s eta 0:00:00
Preparing metadata (setup.py) ... error
error: subprocess-exited-with-error
× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> [11 lines of output]
Traceback (most recent call last):
File "/dentis/.venv/lib/python3.11/site-packages/skbuild/setuptools_wrap.py", line 447, in setup
_check_skbuild_parameters(cmake_install_dir, cmake_source_dir)
File "/dentis/.venv/lib/python3.11/site-packages/skbuild/setuptools_wrap.py", line 280, in _check_skbuild_parameters
raise SKBuildError(msg)
setup parameter 'cmake_source_dir' set to a nonexistent directory.
Project Root : /tmp/pip-install-4j7ztvqi/dracopy_dc9b7455e22b436da9d2969897ccf0fb
CMake Source Directory: ./draco
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.```
