pythongssapi
RFE: provide manylinux binary wheel package
(This is an RFE, not a bug report.)
pip install requests-gssapi pulls in gssapi, and gssapi depends on several development libraries being available. On Fedora and RHEL, it requires gcc, krb5-devel, and the relevant python3-devel package (eg. python3.11-devel if using the python3.11 stack).
In CI containers, we try to keep the installed libraries to a minimum.
Looks like you already ship precompiled wheel files for Mac and Windows. Would you consider doing the same for Linux? (https://pypi.org/project/pysequoia/#files is a good example of doing this.)
Unfortunately this is not really possible for Linux. See https://github.com/pythongssapi/python-gssapi/issues/200 for a more detailed explanation as to why we don't do it for Linux. The reason why we do this for Windows and macOS is because:
- Windows doesn't need to pass the auditwheel requires so can link to a dll that is not provided in the wheel, it requires you to already have the gssapi dll present
- macOS is built against the OS provided GSSAPI library
GSS.Frameworkso we don't have concerns about loading multiple copies of the gssapi library
Thank you for explaining this. The link to #200 helps me understand the history and the bulleted points explain how the Linux situation is different than Windows or macOS. I didn't know that Windows wheels don't have to pass auditwheel.
When Python links to multiple libgssapi_krb5.so libraries (via python-ldap, as explained in #200), what are the problems you would expect to see?
As for your CI problem you have have multiple stages where you build the artifact or your own wheel using one container and share the binaries with the final one. That allows you to avoid having the development headers and build tools present in the final image but still have the compiled module code.
Sure, I think we'll look into that approach for CI.
We also hit this problem regularly on developer workstations. It's difficult for newer developers to get started with Python scripts or apps that use gssapi-enabled web services.
Reviving this topic yet again. Iiuc the only blocker is bundling gssapi and auditwheel? I don't believe the bundling is a strict requirement for manylinux and I did not find wording in the PEP to properly indicate that. If someone has some reference about it please let me know. I also believe that this is not enforced either, most prominently in the GPU wheels.
wheel-next is a project that is meant to help on this front, but it is still difficult to understand how to work with that and it is mostly a responsibility of the build backends to add that support. Setuptools would probably have a though time adding it, but in other backends that integrate with meson/CMake it should be more feasible to support it.
I'm just going to close this as it is not possible to do without potentially introducing hard to debug failures and having to continually update versions of statically linked libraries like OpenSSL. If you really want to have wheels for this package you essentially need to build and host it somewhere else and have it built for your specific environment.
I would have loved to implement this but right now it's just not possible/feasible to do.
without potentially introducing hard to debug failures
That is indeed unavoidable, but that is the case for distro packaged gssapi as well since there is no guaranteed link there either. What you can imply is that it is compatible as far as the gssapi library ABI is compatible.
having to continually update versions of statically linked libraries like OpenSSL
This is the part that I believe is misunderstood from the previous discussions. There is no strict requirement that there is no external library linkage in the wheel, at least I never found the wording to indicate that in the PEPs. For user experience you just need some try...import catching and provide some useful hints.
There are systems though where this breaking fails though. Pypy I believe was one of them and an IBM architecture was another, where the symbol visibility is not guaranteed when a library is dlopen or other edge-cases. But those are challenging to get even simple compiled python bindings to work, and those would be outside of the manylinux wheels anyway.
That is indeed unavoidable, but that is the case for distro packaged gssapi as well since there is no guaranteed link there either. What you can imply is that it is compatible as far as the gssapi library ABI is compatible.
At least currently it fails when trying to install rather than at runtime. It's also complicated due to the different library names for MIT krb5 vs Heimdal. While I would say 99% of Linux users are probably using MIT krb5 that isn't guaranteed and there are packages like Samba which still might use Heimdal.
This is the part that I believe is misunderstood from the previous discussions. There is no strict requirement that there is no external library linkage in the wheel, at least I never found the wording to indicate that in the PEPs. For user experience you just need some try...import catching and provide some useful hints.
Even if there is no strict requirement we cannot guarantee that the required libraries are present at runtime and I really do not want to add runtime checks to try and load things up properly. There are so many permutations just between the various Linux distros and this doesn't include custom environments where some of the default configuration won't work.
Ultimately the effort is just not worth it. If there is an issue where you cannot compile this library at install time you should look at pre-compiling your own wheels that are built for your specific environment setup and storing them in your own repository.