Including pre-compiled (3rd party) binaries for Linux

[K0lb3]

Description

Hello, I would like to include pre-compiled libraries created by a 3rd party (FMOD) in my python module UnityPy. These libraries are closed source, publically available for free, and are being wrapped by my module.

As my module also compiles some C-code by itself, I'm currently facing the issue that cibuildwheel isn't able to publish wheels for Linux, as it detects the 3rd party libraries and errors because of it.

Therefore I would like to know how one should include pre-compiled 3rd party libraries.

Build log

https://github.com/K0lb3/UnityPy/runs/7640536388?check_suite_focus=true#step:6:377

CI config

https://github.com/K0lb3/UnityPy/blob/96174ce883c468f0a4a81cd39417de27e409a7bb/.github/workflows/python-package.yml

[mayeut]

The file https://github.com/K0lb3/UnityPy/blob/master/UnityPyBoost/endian.h is conflicting with the system one. c.f. https://stackoverflow.com/questions/33769348/error-compiling-duplicate-member-on-header-endian-h

If you rename your file, the problem should disappear.

[K0lb3]

Thanks for the reply, but I don't think this will solve the problem. Compiling works fine, the problem is that .so files are found in the source directory.

Checking for common errors...
  Files detected:
    UnityPy/lib/FMOD/Linux/armhf/libfmod.so
    UnityPy/lib/FMOD/Linux/arm/libfmod.so
    UnityPy/lib/FMOD/Linux/x86_64/libfmod.so
    UnityPy/lib/FMOD/Linux/x86/libfmod.so

Error: Command ['python', '-m', 'pip', 'wheel', PurePosixPath('/project'), '--wheel-dir=/tmp/cibuildwheel/built_wheel', '--no-deps'] failed with code 1. 


NOTE: Shared object (.so) files found in this project.

[Czaki]

No. this information about so files is debugging information to help users deal with unwanted artifacts. It was added to cibuildwheel as a response to many personal problems with repair steps because of unwanted files built on the too new system (or some project with linking when different architectures are used).

The real error is here:

     gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/project/UnityPyBoost -I/opt/python/cp310-cp310/include/python3.10 -c UnityPyBoost/AnimationClip.c -o build/temp.linux-x86_64-3.10/UnityPyBoost/AnimationClip.o
    In file included from /usr/include/stdlib.h:42,
                     from /opt/python/cp310-cp310/include/python3.10/Python.h:34,
                     from UnityPyBoost/AnimationClip.h:3,
                     from UnityPyBoost/AnimationClip.c:1:
    /usr/include/bits/waitstatus.h:79:15: error: duplicate member ‘__w_retcode’
       79 |  unsigned int __w_retcode:8;
          |               ^~~~~~~~~~~
    /usr/include/bits/waitstatus.h:80:15: error: duplicate member ‘__w_coredump’
       80 |  unsigned int __w_coredump:1;
          |               ^~~~~~~~~~~~
    /usr/include/bits/waitstatus.h:81:15: error: duplicate member ‘__w_termsig’
       81 |  unsigned int __w_termsig:7;
          |               ^~~~~~~~~~~
    /usr/include/bits/waitstatus.h:93:15: error: duplicate member ‘__w_stopsig’
       93 |  unsigned int __w_stopsig:8; /* Stopping signal.  */
          |               ^~~~~~~~~~~
    /usr/include/bits/waitstatus.h:94:15: error: duplicate member ‘__w_stopval’
       94 |  unsigned int __w_stopval:8; /* W_STOPPED if stopped.  */
          |               ^~~~~~~~~~~
    error: command '/opt/rh/devtoolset-10/root/usr/bin/gcc' failed with exit code 1
    ----------------------------------------
    ERROR: Failed building wheel for UnityPy

[K0lb3]

Nvm, you were right, it worked, so thanks a lot.

In that case I have to say that the log is a bit confusing then, as compiling worked quite fine for me localy.