create_app copies entire contents of `/usr/lib` into `app_name/lib` on archlinux

[julie-is-late]

turns out this behavior isn't exactly desired lol. So the way archlinux packages julia is to install the libs like:

> yay -Ql julia
...
julia /usr/lib/julia/
julia /usr/lib/julia/libccalltest.so
julia /usr/lib/julia/libccalltest.so.debug
julia /usr/lib/julia/libdSFMT.so
julia /usr/lib/julia/libllvmcalltest.so
julia /usr/lib/julia/libsuitesparse_wrapper.so
julia /usr/lib/julia/sys.so
julia /usr/lib/libjulia.so (symlink)
julia /usr/lib/libjulia.so.1 (symlink)
julia /usr/lib/libjulia.so.1.5 (actual library)
...

But in PackageCompiler.jl in create_sysimg_from_object_file the julia install dir is calculated as julia_libdir = dirname(Libdl.dlpath("libjulia")), which because libjulia.so is inside /usr/lib directly causes julia_libdir to get set to /usr/bin/../lib, aka /usr/lib.

Maybe there's a better check to be done about what libraries to copy? Or if arch is breaking the packaging location rules of julia please link me where these are defined so I can take that back to them and have it changed.

[KristofferC]

See the note at https://julialang.github.io/PackageCompiler.jl/dev/#Installation-instructions-1. We could probably try harder to detect when this isn't the case.

[MasFlam]

I seem to have a similar problem on Debian, where the julia libs got installed by apt into /lib/x86_64-linux-gnu/julia/ and the libjulia.so file is on the same level as the julia/ directory. And similarly to what @jshap70 experienced, the entirety of that directory gets copied over to MyPackageBuild/lib, making the build weigh over 2GiB.

[MasFlam]

But it seems like it's the same issue - copying one directory higher unnecessarily

[KristofferC]

See the note at https://julialang.github.io/PackageCompiler.jl/dev/#Installation-instructions-1. We could probably try harder to detect when this isn't the case.

[wxyangf]

Hi, just wrote a hacky patch for this, very not tested.

function bundle_julia_libraries(app_dir)
    app_libdir = joinpath(app_dir, Sys.isunix() ? "lib" : "bin")
    fl = readdir(julia_libdir())
    libs = [julia_libdir()*"/"*fl[i] for i in findall(X->occursin("julia", X), fl)]

    println(app_libdir)
    mkpath(app_libdir)

    for l in libs
        cp(l, app_libdir*"/"*split(l,"/")[end])
    end

    # We do not want to bundle the sysimg (nor the backup sysimage):
    rm(joinpath(app_libdir, "julia", default_sysimg_name()); force=true)
    rm(joinpath(app_libdir, "julia", backup_default_sysimg_name()); force=true)
    # Remove debug symbol libraries
    if Sys.isapple()
        v = string(VERSION.major, ".", VERSION.minor)
        rm(joinpath(app_libdir, "libjulia.$v.dylib.dSYM"); force=true, recursive=true)
        rm(joinpath(app_libdir, "julia", "sys.dylib.dSYM"); force=true, recursive=true)
    end
    return
end

Basically it tries to be a bit smarter and grab only julia relevant things, it will prob fail on systems that have different julia packaging (eg. not just dumped in the lib folder)

A better patch would also look up in the filesystem to see if the julia_libdir is named "julia" and if so just flatcopy everything.

[KristofferC]

This is also the case if you use a locally built Julia, then you get a bunch of object files bundled.

[originalsouth]

Another work around:

1. download the latest Julia here https://julialang.org/downloads/
2. unpack it to say /path/to/unpacked/
3. run /path/to/unpacked/julia-*/bin/julia buildscript.jl