Install stable ABI library in pydebug builds (`libpython3d.so`)

[Osyotr]

Bug report

Release and debug builds of python are now ABI-compatible (https://github.com/python/cpython/pull/12615 https://github.com/python/cpython/pull/12952), so there is no need to skip build of libpython3.so when building with --with-pydebug flag. https://github.com/python/cpython/blob/b24e8b28a7dc585ba367a959be83393f2352d21d/configure.ac#L1498-L1501

Your environment

• CPython versions tested on: 3.10.5
• Operating system and architecture: Debian Linux x64

[corona10]

cc @vstinner @encukou

[vstinner]

Sorry, but I don't know this libpython3.so file nor its purpose.

When Python is built with ./configure --enable-shared, the python program is a very tiny program linked to libpython just to call Py_BytesMain(argc, argv), that's it. But it's linked to a filename like libpython3.10.so.1.0 (on Fedora), not libpython3.so.

When Python is built in debug mode, the libpython filename is different because the implementation is different. For example, on Fedora, python3-debug is linked to libpython3.10d.so.1.0: the libpython filename gets an additional d suffix.

While the ABI is compatible, the implementation remains different and so the libpython filename must be different. IMO that's why libpython3.so is not created in debug mode.

If you want to have a libpython3.so built in debug mode, IMO it should have different filename.

Would you mind to elaborate your useful. What are you trying to do?

[Osyotr]

Sorry, but I don't know this libpython3.so file nor its purpose.

This is so-called "Python Stable ABI" https://peps.python.org/pep-0384/#linkage:

If Python is compiled as a shared library, it is installed as both libpython3.so, and libpython3.y.so; applications conforming to this PEP should then link to the former.

If you want to have a libpython3.so built in debug mode, IMO it should have different filename.

Absolutely.

Would you mind to elaborate your useful. What are you trying to do?

This is not really a problem, just an inconvenience when packaging python in vcpkg. Since the number of binaries differs between release and debug builds, the check for it is disabled. This can lead to hard-to-spot errors if for example there's some module that haven't been built successfully.

• https://github.com/microsoft/vcpkg/pull/25995

[encukou]

I guess libpython3d.so would make sense for consistency, but as Victor says it's not useful in practice. Applications that need to link to an alternate library for debugging can link directly to libpython3.12d.so.

You can make libpython3.so in the debug build today, though it isn't built and installed by default.

IMO the way libpython3.so is made today isn't ideal -- either it could just be just a symlink to libpython3.12.so (as PEP-384 intended, IMO), or it should actually only re-export a limited set of symbols, like on Windows (this would be an improvement to the initial implementation specified in the PEP, and it probably wouldn't be trivial to get there).

[vstinner]

PEP 384:

On Windows, applications shall link with python3.dll; an import library python3.lib will be available. This DLL will redirect all of its API functions through /export linker options to the full interpreter DLL, i.e. python3y.dll.

Can we update PEP 384 to not do that anymore? Since Python 3.8, C extensions are no longer linked to libpython on purpose.

Example on Fedora 36:

$ ldd /usr/lib64/python3.10/lib-dynload/_asyncio.cpython-310-x86_64-linux-gnu.so
	linux-vdso.so.1 (0x00007fff57de2000)
	libc.so.6 => /lib64/libc.so.6 (0x00007fbdcf04f000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fbdcf277000)

The _asyncio extension is not linked to libpython.

[encukou]

PEP 384 is a historical document, but we could update the implementation. But, as far as I know, what PEP 384 says makes sense for Windows.

[encukou]

Let me close this. It could be solved by adding libpython3d.so, but that doesn't seem useful. Or it could be solved by removing libpython3.so, butsomeone might be depending on it. Either way, it should get a wider discussion than on this issue – e.g. on Discourse.

And it's not a bug, but more of a distribution linter warning. I think I understand where it's coming from (Visual Studio has much more consistent handling of Release/Debug builds than Makefiles, so there this would indicate a build issue?). But without a real use case, I don't see us making this a priority.