pypa
Running tests on vanilla system?
Description
I'm just in the process of extending our Python bindings for Valhalla (C++ project) and discovered super late that auditwheel misses patching the SONAME of one transitive dependency of the final _valhalla.xxx.so library (see auditwheel issue).
Our CIBW_TEST_COMMAND is simply python -c 'from valhalla import Actor' with the intention to catch exactly that, smth like linker problems (the real tests for the bindings are run in a different CI workflow). The Valhalla C++ project is dependent on quite a lot of libraries, which we install into our own manylinux fork. So we end up with all the dependency .so's installed system-wide and when running the CIBW_TEST_COMMAND it finds them, even if auditwheel misses to re-write their ELF entries to the package-local, mangled .so. But in a fresh python container and only installing the pyvalhalla package, we end up with linker errors.
Would it be an idea to run the cibuildwheel test step in a similar isolated environment like a python docker container, so these linker problems can be caught? Note, I have very little idea about the code base of cibuildwheel so far. But I'd be open to contribute if that'd be considered ok to add. Implementing projects can of course check that themselves too before uploading to PyPI, but it was quite a gotcha for me that I missed to consider and would have loved cibuildwheel to tell me early on.
Build log
No response
CI config
No response
It's a question of additional complexity - is it going to be worth carrying that functionality forward forever. It's also only really possible on linux, the other platforms don't have that luxury.
I wonder if a complementary tool citestwheel with a similar options interface would be good - that way it could be used in a CI setting on a separate runner so all platforms could benefit. I'm not volunteering to build it, I've got enough keeping me busy these days. Would be happy to advise if somebody else thought it was a good idea.
