Python bindings, consider to get rid of ###LIBLOUIS_SONAME### macro

[LeonarddeR]

The liblouis bindings now require to be build, and the name of the library is hardcoded in it (i.e. liblouis.so.16). However, I don't think this is necessary.

1. On initializing the library, we already check whether to use WinDLL or CDLL. WinDLL implies the .dll extension, CDLL implies the .so extension
2. I assume that on most linux systems, it is sufficient to load liblouis.so instead of liblouis.so.16 or even liblouis.so.16.0.0. usually, liblouis.so is symlinked to the most recently installed version right?
3. It allows the distribution of the python bindings within pypi. This sounds like a nasty thing to do, but it makes perfect sense to me if you take into account that Liblouis can be installed as part of the Windows 10 braille system, in which case liblouis will be installed in system32 and can be loaded out of the box. In this case, you can install just the python bindings and control the Windows 10 bundles liblouis. Note that I see this point merely as a nice to have.

[mwhapples]

Problems with this.

1. cdll does not automatically mean .so, think about MacoSX with .dynlib, there may be other platforms.
2. Is this really a good assumption? The bindings are for a specific version. If the user has multiple versions installed and the python bindings belong to the older version the assumption could lead to software breakage. I think linking to the intended version is safest.
3. Based on 2, should a pypi bundle include the LibLouis library as well to make it batteries included and to ensure the correct version. However I think there are some packages on pypi which don't include binary libraries and rely on them being installed on the system, so that may just be a preference thing.

I agree with removing the macro use as it means the python setup.py cannot be used by itself, requires the use of the make tools, which is not really the python way. I just feel the assumptions are incorrect and will cause problems.

[LeonarddeR]

@mwhapples commented on 6 sep. 2018 12:42 CEST:

1. cdll does not automatically mean .so, think about MacoSX with .dynlib, there may be other platforms.

Hmm yes, that makes perfect sense. I wonder whether there is a way to get this extension.

2. Is this really a good assumption? The bindings are for a specific version. If the user has multiple versions installed and the python bindings belong to the older version the assumption could lead to software breakage. I think linking to the intended version is safest.

I agree that this is the safest method to make sure that you never have a missmatch. Having said that though, it also makes sense not to introduce backwards incompatible API changes within a major version of liblouis.

3. Based on 2, should a pypi bundle include the LibLouis library as well to make it batteries included and to ensure the correct version. However I think there are some packages on pypi which don't include binary libraries and rely on them being installed on the system, so that may just be a preference thing.

In case of binding the bindings to a specific version, bundling the library makes sense.

I agree with removing the macro use as it means the python setup.py cannot be used by itself, requires the use of the make tools, which is not really the python way.

Do you have alternative ideas to get rid of it?

[egli]

The current implementation using the ###LIBLOUIS_SONAME### is based on a discussion on the mailing list between @sthibaul , @jcsteh and @mwhapples . I remember the three of them hashed it out and came to the current solution.

Seems like there are reasons to hard code the soname.

[mwhapples]

This is where a Cython or Python C extension works better as they get linked like any other C or C++ application would and so would not require this macro in the Python code. However without rewriting the Python bindings, this is not where we are an we need to find this at runtime for ctypes.

Whilst I may not like the use of the macro, it does solve some problems and possibly in a moderately simple way. There are possibly some things which could be done to avoid this, but I am not sure whether they really are much better, except for meaning that the python bindings could be built separate from LibLouis.

1. Rather than using windll and cdll to determine the extension of shared libraries, it would be better to use the platform/OS information. Windll and cdll are really the calling system used and are separate from the extension (based on the comments in the bindings cygwin uses cdll but I think uses the dll extension). NOTE: The bindings currently don't use windll and cdll to determine the extension, responding to the suggestion of using them for that.
2. API backwards compatability only works so far. What happens when the user has two API different versions installed (eg. 4.0 and 3.7.0)? Using a function to get the version only goes so far as it tells the bindings they have the wrong version but offers nothing to find the correct version. For JLouis I had it that it can use some system properties to instruct JLouis what file to load as the library. Not sure Python has anything the same as Java's system properties, may be environment variables could be used. Still not really the full solution but would help people instruct the bindings when the default behaviour fails.

[whentze]

Something to note:

2. I assume that on most linux systems, it is sufficient to load liblouis.so instead of liblouis.so.16 or even liblouis.so.16.0.0. usually, liblouis.so is symlinked to the most recently installed version right?

This is not quite true. Most distros (all the major ones besides arch, if I recall correctly) do not ship libxyz.so symlinks in the library packages themselves. Instead, that symlink is installed by the libxyz-dev package. While most people hacking on liblouis will have liblouis-dev installed anyway, this is not the case for end-users.

[sthibaul]

Note: the question between major versions is not API, but ABI. API incompatibility is really a burden since it requires software using liblouis to be fixed. ABI incompatibility is much less a concern since it just requires rebuilding the software using liblouis. Ideally ABI does not change to avoid the rebuild, but sometimes you have to do it. And then you really need to have something that distinguishes the two .so, .dll, or .dylib, otherwise you get ABI incompatibilities which result in run-time breakage. That's actually exactly the reason why Linux distros do not ship the .so file in the library package, but only in the development library package.'

The fact that windows would ship it out the box does not change that ABI issue. Dlls provided by windows either have a version (see msvc*.dll), or have an ABI that shall never change, which we probably can't promise with liblouis.