heir icon indicating copy to clipboard operation
heir copied to clipboard

fix lld/libomp misconfig in cibuildwheel

Open j2kun opened this issue 3 months ago • 6 comments

The forcing of the lld linker is breaking the cibuildwheel, and once I fixed that I found it was broken by a lack of openmp. I found a way to allow libomp and libgomp by using just the -fopenmp option, which allows the compiler to pick the exact linker flag based on what's available on the system. Also needed to install libgomp in the cibuildwheel container.

j2kun avatar Sep 22 '25 19:09 j2kun

Going to run cibuildwheel locally to test this before merging it, on top of Alex's changes to the python deps

j2kun avatar Sep 22 '25 20:09 j2kun

ok so we actually have a slight problem here in that the manylinux specification forbids linking against openmp (linking against any shared lib not in this list implies noncompliance: https://peps.python.org/pep-0599/#the-manylinux2014-policy)

I think to get this to work now I have to bundle libgomp/libomp with the wheel, and then point to it in during the clang compile step. Or just ship heir-py with openmp disabled.

j2kun avatar Sep 23 '25 03:09 j2kun

Specifically cibuildwheel builds and then fails at

Repairing wheel...

    + sh -c 'auditwheel repair -w /tmp/cibuildwheel/repaired_wheel /tmp/cibuildwheel/built_wheel/heir_py-0.0.2-cp311-abi3-linux_x86_64.whl'
INFO:auditwheel.main_repair:Repairing heir_py-0.0.2-cp311-abi3-linux_x86_64.whl
auditwheel: error: cannot repair "/tmp/cibuildwheel/built_wheel/heir_py-0.0.2-cp311-abi3-linux_x86_64.whl" to "manylinux_2_28_x86_64" ABI because of the presence of too-recent versioned symbols. You'll need to compile the wheel on an older toolchain.

Repro (from a venv with cibuildwheel installed):

CIBW_DEBUG_KEEP_CONTAINER=1 cibuildwheel --platform linux --archs x86_64

Then after it fails, docker exec -it <container_id> /bin/bash, and in the container:

$ /opt/python/cp311-cp311/bin/python -m pip install auditwheel-symbols
$ /opt/python/cp311-cp311/bin/auditwheel-symbols --manylinux 2_28  /tmp/cibuildwheel/built_wheel/heir_py-0.0.2-cp311-abi3-linux_x86_64.whl

heir/abc is manylinux_2_28 compliant.
heir/heir-opt is not manylinux_2_28 compliant because it links the following forbidden libraries:
libgomp.so.1
heir/heir-translate is manylinux_2_28 compliant.
heir/libOPENFHEbinfhe.so is not manylinux_2_28 compliant because it links the following forbidden libraries:
libc.so.6       offending versions: GLIBC_2.32
libstdc++.so.6  offending versions: GLIBCXX_3.4.29
libm.so.6       offending versions: GLIBC_2.29
heir/libOPENFHEcore.so is not manylinux_2_28 compliant because it links the following forbidden libraries:
libc.so.6       offending versions: GLIBC_2.32
libm.so.6       offending versions: GLIBC_2.29
libstdc++.so.6  offending versions: GLIBCXX_3.4.29
heir/libOPENFHEpke.so is not manylinux_2_28 compliant because it links the following forbidden libraries:
libm.so.6       offending versions: GLIBC_2.29
libstdc++.so.6  offending versions: GLIBCXX_3.4.29, GLIBCXX_3.4.26
libc.so.6       offending versions: GLIBC_2.32
heir/libopenfhe.so is not manylinux_2_28 compliant because it links the following forbidden libraries:
libgomp.so.1
heir_py-0.0.2.data/data/bin/heir-opt is not manylinux_2_28 compliant because it links the following forbidden libraries:
libgomp.so.1
heir_py-0.0.2.data/data/bin/heir-translate is manylinux_2_28 compliant.

j2kun avatar Sep 23 '25 03:09 j2kun

The errors like libc.so.6 offending versions: GLIBC_2.32 I can fix by bumping to a newer manylinux container version, but the libgomp error would not be fixed by this.

j2kun avatar Sep 23 '25 03:09 j2kun

Just to unblock releases at this point, I'm going to disable openmp in the heir-py build and file an issue.

j2kun avatar Sep 23 '25 03:09 j2kun

Another option: https://github.com/cerisier/toolchains_llvm_bootstrapped

j2kun avatar Sep 23 '25 03:09 j2kun