numpy build fails

[smartsammler]

Hello,

Thank you for this nice project. I tried to use it for building debian/ubuntu packages in a CI/CD pipeline for ubuntu focal. Unfortunately numpy does not work out of the box.

For me it looks like wheel2deb has some problems with manylinux2010, because firstly, older versions of numpy (==1.18.5) where a manylinux1 wheel is downloaded when invoking pip3 wheel numpy==1.18.5 work. Secondly, the manylinux2010 wheel works with pip (pip install numpy).

(Only tested interactively, downloading the current manylinux1-wheel and running wheel2deb works, too.)

A small example that shows my problem can be created with the following dockerfile

# Dockerfile
FROM ubuntu:focal

ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update \
    && apt-get install -y --no-install-recommends \
    git \
        python3-pip \
        python3-apt \
        apt-file \
        dpkg-dev \
        fakeroot \
        build-essential \
        devscripts \
        debhelper \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/*
ENV DEBIAN_FRONTEND=

# Install the newest wheel2deb package which fixes some versioning issues.
RUN python3 -m pip install -U git+https://github.com/upciti/wheel2deb.git@master#egg=wheel2deb

COPY failing_test.sh /failing_test.sh

CMD ["/usr/bin/env", "bash", "/failing_test.sh"]

and the bash script that runs the actual failing test cases:

# failing_test.sh
#!/usr/bin/env bash

python3 -m pip wheel numpy
ls *
# numpy-1.20.1-cp38-cp38-manylinux2010_x86_64.whl
mkdir output
wheel2deb
wheel2deb build
dpkg -i output/*.deb

python3 -m pip freeze
echo ""
python3 -c "import numpy" || echo "manylinux 2010 doesn't work via wheel2deb"
echo ""

# But with pip it works
apt remove -y python3-numpy
python3 -m pip install numpy==1.20.1
echo ""
python3 -c "import numpy" && echo "manylinux 2010 works via pip"
echo ""

With these two files and docker one can run

sudo docker build -t wheel2deb-err -f Dockerfile .
sudo docker run -ti wheel2deb-err:latest

to reproduce the error.

Unfortunately, numpy is going to drop support for manylinux1 soon https://github.com/pypa/manylinux/issues/542#issuecomment-720164102 so trying to work around this problem by using the manylinux1 wheels does not work in the long run.

[smartsammler]

Sorry, I totally forgot to paste the actual error message. For completeness, here it is:

Python 3.8.5 (default, Jan 27 2021, 15:41:15) 
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/numpy/core/__init__.py", line 22, in <module>
    from . import multiarray
  File "/usr/lib/python3/dist-packages/numpy/core/multiarray.py", line 12, in <module>
    from . import overrides
  File "/usr/lib/python3/dist-packages/numpy/core/overrides.py", line 7, in <module>
    from numpy.core._multiarray_umath import (
ImportError: libgfortran-2e0d59d6.so.5.0.0: ELF load command address/offset not properly aligned

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3/dist-packages/numpy/__init__.py", line 145, in <module>
    from . import core
  File "/usr/lib/python3/dist-packages/numpy/core/__init__.py", line 48, in <module>
    raise ImportError(msg)
ImportError: 

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.

We have compiled some common reasons and troubleshooting tips at:

    https://numpy.org/devdocs/user/troubleshooting-importerror.html

Please note and check the following:

  * The Python version is: Python3.8 from "/usr/bin/python3"
  * The NumPy version is: "1.20.1"

and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.

Original error was: libgfortran-2e0d59d6.so.5.0.0: ELF load command address/offset not properly aligned

[fyhertz]

Hi,

Thank you for taking the time to point this out. It seems that dh_strip messes up libgfortran for some reason.

Adding this:

override_dh_shlibdeps:
        dh_strip --exclude=libgfortran

To python3-numpy_1.20.1-1~w2d0_amd64/debian/rules solves the issue.

I guess I should add an option to exclude libraries from dh_strip or just disable it altogether in wheel2deb to prevent such issues.

[smartsammler]

Hi,

Thank you for helping. Unfortunately, Your advice does not work for me. After running wheel2deb, I changed output/python3-numpy_1.20.1-1~w2d0_amd64/debian/rules, so that now it looks like

#!/usr/bin/make -f

%:
        dh $@

override_dh_shlibdeps:
        dh_strip --exclude=libgfortran

and then ran wheel2deb build and dpkg -i output/numpy*.deb, and still get the same error message in python3.