After built with pyapp, pip install failed for hdbscan

[sandangel]

After built with pyapp, calling pyapp self restore will fail because it could not install a dependency: hdbscan

gcc -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64-v3 -fPIC -fPIC -I/root/.local/share/pyapp/myapp/2930268585031877112/2.0.5/python/include/python3.11 -I/tmp/pip-build-env-8_vyixci/overlay/lib/python3.11/site-packages/numpy/core/include -c hdbscan/_hdbscan_tree.c -o build/temp.linux-x86_64-cpython-311/hdbscan/_hdbscan_tree.o

Please note that the -march=x86-64-v3, which is not correct, should be x86-64. Without packaging with pyapp, I can install with normal pip without issue. Another issue is it does not detect gcc, but will require clang

FROM rust:1-debian
ENV PYAPP... # similar to cowsay, expose pip
WORKDIR /app
RUN cargo install pyapp --root /app
RUN pyapp pip install hdbscan # <-- fail

Workaround

FROM rust:1-debian
ENV PYAPP... # similar to cowsay, expose pip
WORKDIR /app
RUN cargo install pyapp --root /app
RUN CC=gcc CFLAGS="-march $(uname -m | tr '_' '-')" pyapp pip install hdbscan # <-- ok

[ofek]

Can you please try rebuilding with one of the variant options? https://ofek.dev/pyapp/latest/config/#cpython

[sandangel]

Let me give it a try

[sandangel]

@ofek it doesn't work too. now the command change to:

clang -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64-v4 -fPIC -fPIC -I/root/.local/share/pyapp/myapp/15114544842415842793/2.0.5/python/include/python3.11 -I/tmp/pip-build-env-d572aapz/overlay/lib/python3.11/site-packages/numpy/core/include -c hdbscan/_hdbscan_tree.c -o build/temp.linux-x86_64-cpython-311/hdbscan/_hdbscan_tree.o

Notice the -march now became x86-64-v4.

[ofek]

Try v1

[sandangel]

@ofek v1 does not have -march=x86-64-v1 issue, however, it still does not detect gcc and still requires clang. I tried v2 v3 v4, will have both issues.

[ofek]

I think it might be an issue with that particular package:

• https://github.com/scikit-learn-contrib/hdbscan/issues/525
• https://github.com/scikit-learn-contrib/hdbscan/issues/600

As far as why that first error is gone, it's related to the CPU architecture https://gregoryszorc.com/docs/python-build-standalone/main/running.html

I don't know exactly what's happening in your case but would it help if you could set runtime environment variables at build time?

[sandangel]

hi @ofek , I'm not sure what do you mean by setting runtime env var at build time?

I don't think the problem is with that particular package because I was able to install it using normal python, not python from the python distribution. I also tried external pip, but it still does not work. I guess I will need to keep using the CC=gcc flag?

[ofek]

Can you please give me more details about your environment?

What I meant was would it be useful to you if you could inject environment variables into the binary such that they would be available at runtime like CC?

[sandangel]

@ofek oh, I see. I'm building docker image like the example above. So you mean I can use:

ENV CC=gcc

To set the env var? That does not fix the issue that python from distribution does not detect gcc automatically, and I will have to use cpu v1 for the portability. I'm not sure if it will affect performance.

[sandangel]

Hi, now I got another error:

No default distribution source found

I'm using aarch64 linux, so it should be supported.

[ofek]

Show the full error?

[sandangel]

Caused by:
  process didn't exit successfully: `/tmp/cargo-installSIy4m9/release/build/pyapp-ce661a8f58de8e66/build-script-build` (exit status: 101)
  --- stdout
  cargo:rustc-env=PYAPP_PROJECT_DEPENDENCY_FILE=
  cargo:rustc-env=PYAPP__PROJECT_DEPENDENCY_FILE_NAME=
  cargo:rustc-env=PYAPP_PROJECT_NAME=wovey
  cargo:rustc-env=PYAPP_PROJECT_VERSION=2.0.5
  cargo:rustc-env=PYAPP__PROJECT_EMBED_FILE_NAME=wovey-2.0.5-py3-none-any.whl
  
  --- stderr
  thread 'main' panicked at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/pyapp-0.16.0/build.rs:364:5:


  No default distribution source found
  Python version: 3.11
  Platform: linux
  Architecture: aarch64
  ABI: gnu
  Variant: v1


  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
error: failed to compile `pyapp v0.16.0`, intermediate artifacts can be found at `/tmp/cargo-installSIy4m9`.
To reuse those artifacts with a future compilation, set the environment variable `CARGO_TARGET_DIR` to that path.
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/root/.local/share/hatch/env/virtual/wovey/9TtSrW0h/wovey-build/lib/python3.11/site-packages/hatchling/__main__.py", line 6, in <module>
    sys.exit(hatchling())
             ^^^^^^^^^^^
  File "/root/.local/share/hatch/env/virtual/wovey/9TtSrW0h/wovey-build/lib/python3.11/site-packages/hatchling/cli/__init__.py", line 26, in hatchling
    command(**kwargs)
  File "/root/.local/share/hatch/env/virtual/wovey/9TtSrW0h/wovey-build/lib/python3.11/site-packages/hatchling/cli/build/__init__.py", line 82, in build_impl
    for artifact in builder.build(
  File "/root/.local/share/hatch/env/virtual/wovey/9TtSrW0h/wovey-build/lib/python3.11/site-packages/hatchling/builders/plugin/interface.py", line 155, in build
    artifact = version_api[version](directory, **build_data)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/share/hatch/env/virtual/wovey/9TtSrW0h/wovey-build/lib/python3.11/site-packages/hatchling/builders/binary.py", line 163, in build_bootstrap
    self.cargo_build(install_command, cwd=context_dir, env=env)
  File "/root/.local/share/hatch/env/virtual/wovey/9TtSrW0h/wovey-build/lib/python3.11/site-packages/hatchling/builders/binary.py", line 195, in cargo_build
    raise OSError(message)
OSError: Compilation failed (code 101)
Error: building at STEP "RUN PYAPP_PROJECT_PATH=/app/dist/wovey-$(rye version)-py3-none-any.whl hatch build --target binary": while running runtime: exit status 1

[sandangel]

The same dockerfile built on x86_64, but failed locally on aarch64

[ofek]

The variant environment variable I think is the issue for ARM, can you try removing that?

[sandangel]

@ofek , yeah, it looks like variant v1 does not exist for aarch64. It works when I use v3 default. However, v3 make the wheel build error on x86_64, and require CFLAGS. But the CFLAGS does not work on aarch64. :(

[sandangel]

Ended up setting CFLAGS for now.