BinaryProvider doesn't detect that FreeBSD already has what's needed for CodecXz

[ko56]

(Related to issues #32 and #57) My suggestion is to have the build.jl script, here we're using CodecXz as an example, somehow check whether the library it's looking for already exists on the system, and if so, proceed directly to creating the deps.jl file. In this specific case, FreeBSD has /usr/lib/liblzma.so.

I made CodecXz work by manually creating a deps.jl file with the right path to the library.

[ararslan]

This is expected, since it's how BinaryProvider is designed to work.

For some background, one problem you run into when using system libraries instead of building the libraries yourself is versioning: the version of a particular library installed on the system may be very old or otherwise mismatched/incompatible with what you're building for other systems, which can cause differences in the behavior of the Julia package which calls it across systems. That's part of the motivation for BinaryBuilder and BinaryProvider, that system libraries should be avoided. It's easier and more consistent to just let the builder build for FreeBSD.

[ko56]

I understand your point about versions. However, (a) sometimes building a package for a particular OS requires specific tricks, and I don't know how BinaryProvider deals with that, and (b) in this, the CodecXz case, the builder responds that FreeBSD is an unknown platform.

[ararslan]

CodecXz likely just needs a new tag then, since XzBuilder has been updated to support FreeBSD: https://github.com/bicycle1885/XzBuilder/commit/82d8f6db15930f4be2fa598c92d86131b81ef8e3#diff-2567705d891d25120e6ea4ca97383113R34

[klpn]

In some cases, this avoidance of system libraries gives rise to new problems with versioning. For example, Arch Linux repos provide both Julia and Arpack, compiled with the version of GCC in the Arch repos (GCC 8). But if you try to install Arpack.jl, the build tries to download and use a version of Arpack created with GCC 7, which does not work with the GCC 8-compiled Julia. As I described under Arpack.jl #5, it is possible to get around the problem by manually copying the Arpack system library into the build directory, but this is not a very attractive solution. Perhaps, the build.jl could some decision method like Pypandoc which uses the previously installed Pandoc if it is not older than the provided Pandoc. One could also argue that the build_tarballs.jl scripts generated by BinaryBuilder should use static linking as far as possible.

[ko56]

I think this is a good suggestion "Perhaps, the build.jl could some decision method like Pypandoc which uses the previously installed Pandoc if it is not older than the provided Pandoc". It would also mitigate the tendency to create an "overly fat" Julia installation.

[ararslan]

Arch Linux repos provide both Julia and Arpack

We always recommend using official Julia binaries over ones provided by system package managers. That's mentioned in Julia's README.

[klpn]

Yes, it says:

Although some system package managers provide Julia, such installations are neither maintained nor endorsed by the Julia project. They may be outdated and/or unmaintained.

In the case with Arch, however, the package manager provides the latest released version of Julia built with the latest released version of the GCC toolchain. I was wondering how the build should behave in such cases.