BinaryBuilder.jl
BinaryBuilder.jl copied to clipboard
Help relocatability of libraries/executables in the JLL wrappers
We have several cases where we need to set some environment variables at runtime (or using withenv
) to relocate the libraries/executables:
- for
git
we need to setGIT_EXEC_PATH
andGIT_SSL_CAINFO
, see https://github.com/JuliaPackaging/Yggdrasil/issues/284 and https://github.com/JuliaPackaging/Yggdrasil/issues/452 - for
ncurses
we need to setTERMINFO
orTERMINFINO_DIRS
, see https://github.com/JuliaPackaging/Yggdrasil/issues/455 - for
Tk
/Tcl
we need to setTCL_LIBRARY
andTK_LIBRARY
, see https://github.com/JuliaPackaging/Yggdrasil/issues/462 - for
OpenMPI
we need to setOPAL_PREFIX
, see https://github.com/JuliaPackaging/Yggdrasil/issues/390 - for
Gtk
we need to set a bunch of environment variables, see the__init__
function ofGtk.jl
- for
Fontconfig
we need to setFONTCONFIG_FILE
, see the__init__
functions ofFontconfig.jl
andCairo.jl
We usually delegate the Julia packages calling the libraries to set these variables, but this has a couple of drawbacks:
- all packages calling the JLL package need to do it, see the example of
Cairo.jl
andFontconfig.jl
, both usingFontconfig_jll
- in some cases, the Julia packages doesn't need to know what JLL package it's using. This is for example the case of
MPI.jl
reported by @simonbyrne in https://github.com/JuliaPackaging/Yggdrasil/issues/390
Maybe we should allow setting these variables directly in the JLL wrappers, to alleviate the burden of the developers of the high-level interface of the package.
The problem is that sometimes these variables need to be set around an executable -- so that we could add these to the already existing wrapper -- but sometimes we need to define the variable globally because it's used by a library, so deciding an interface to do this may not be very easy.
I'm wondering if it doesn't make sense to try and carry patches that make these kinds of shenanigans unnecessary.
In most cases patching the source code might work, but it's more burden on our side to prepare the patches and maintain them in the long term. However, for git we need to set
ENV["GIT_SSL_CAINFO"] = joinpath(dirname(Sys.BINDIR), "share", "julia", "cert.pem")
I don't think we can escape setting some sort of environment variable in this case.
It would be really useful to go beyond just allowing to set a few custom variables: how about making it possible to inject some arbitrary extra initialization code? E.g. for libraries like GMP, FLINT and others, one might want to initialize memory management functions. Right now, every caller of such a library has to set these (or hope somebody else did it).
A very simple solution would be a way to specify a custom __init__
function for the _jll
files (say as a quote
block passed to build_tarballs
)
Based on a comment by @staticfloat on PR #791, I had a look at which env var values are in fact in use. Here is what I got (based on @giordano's helpful list):
- [ ] for git (see JuliaPackaging/Yggdrasil#284 and JuliaPackaging/Yggdrasil#452)
-
GIT_EXEC_PATH = joinpath(Git_jll.artifact_dir, "libexec", "git-core")
-
GIT_SSL_CAINFO = joinpath(dirname(Sys.BINDIR), "share", "julia", "cert.pem")
-
- [ ] for ncurses we need to set one of these (see JuliaPackaging/Yggdrasil#455)
-
TERMINFO = joinpath(Ncurses_jll.artifact_dir, "share", "terminfo")
-
TERMINFO_DIRS = joinpath(Ncurses_jll.artifact_dir, "share", "terminfo")
-
- [ ] for Tk/Tcl (see JuliaPackaging/Yggdrasil#462)
-
TCL_LIBRARY = joinpath(Tcl_jll.artifact_dir, "lib", "tcl8.6")
-
TK_LIBRARY = joinpath(Tk_jll.artifact_dir, "lib", "tk8.6")
-
- [x] for OpenMPI (see JuliaPackaging/Yggdrasil#390)
-
OPAL_PREFIX = OpenMPI_jll.artifact_dir
-
- [x] for Fontconfig we need to set FONTCONFIG_FILE, see the
__init__
functions of Fontconfig.jl and Cairo.jl-
FONTCONFIG_FILE = joinpath(Fontconfig_jll.artifact_dir, "etc", "fonts", "fonts.conf")
-
- [ ] for Gtk we need to set a bunch of environment variables, see the init function of
Gtk.jl
-
XDG_DATA_DIRS
-- complex value, depends on multiple JLLs -
GDK_PIXBUF_MODULE_FILE
-- depends on a mutable artifact -
GDK_PIXBUF_MODULEDIR
-
- [x]
Xorg_xkeyboard_config
-
XKB_CONFIG_ROOT
(but only on Linux and FreeBSD?)
-
Based on the above, it seems that setting env variables to a path relative to the artifact_dir
would cover most
cases; the exceptions in the list above being
-
GIT_SSL_CAINFO
, which is set to a path relative to the Julia install path - Gtk.jl, which however all in all seems out of scope here anyway
There is of course a good chance that there are other relevant env vars out there which
also don't fit into the "relative to artifact_dir
" scheme, but it still seems as if
adding a way to specify such env vars be set in JLLs would be useful.
That said, it might not be completely without issue: e.g. setting TERMINFO
could have
ripple effects: it would e.g. affect subprocesses launched by Julia, which then might
break (because those subprocesses might run executable which are linked against a different
copy of ncurses and which expect require a different TERMINFO
value to work right).
So I am not 100% sure how to resolve that (or perhaps nothing needs to be resolved and
I am just overthinking it?)
I am encountering this in PMIx_jll and pprte_jll which both copy OpenMPI MCA approach. It is again an environment variable relative to the artifact_dir.
init_block
was added in #791, it appears to be used in a couple of recipes in Yggdrasil.