Rmath-julia
Rmath-julia copied to clipboard
Published
20 hours ago •
JuliaStats
JuliaStats
libgcc_s.1.dylib dependency of libRmath-julia.dylib in Rmath_jll cannot be resolved
Hi, thank you for this useful package. I'm using "x86_64-apple-darwin14" and found that libgcc_s.1.dylib dependency of libRmath-julia.dylib in Rmath_jll could not be resolved. Specifically, the libgcc_s.1.dylib dicrectory: /workspace/destdir/x86_64-apple-darwin14, seems inappropriate and results in an error when using a package, that depends on Rmath_jll (RCall in my case).
$ otool -L ~/.julia/artifacts/d779ae56a8eaabc3dc8e0f2ab17907389b96d155/lib/libRmath-julia.dylib
/Users/USERNAME/.julia/artifacts/d779ae56a8eaabc3dc8e0f2ab17907389b96d155/lib/libRmath-julia.dylib:
@rpath/libRmath-julia.dylib (compatibility version 0.0.0, current version 0.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1238.0.0)
/workspace/destdir/x86_64-apple-darwin14/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)
I may find a way to fix this in another Julia package in Yggdrasil: https://github.com/JuliaPackaging/Yggdrasil/blob/13cad4792b456b2536753955fec85448d9917df9/C/CompilerSupportLibraries/common.jl#L23. Unfortunately, however, I'm very new to Julia, and I'm sorry for that I couldn't submit a pull request. I'd really appreciate it if this would be fixed in a future release.
I am seeing this with a clean install of R/Julia:
> JuliaCall::julia_setup()
Julia version 1.10.0 at location /Applications/Julia-1.10.0.app/Contents/Resources/julia/bin will be used.
Loading setup script for JuliaCall...
ERROR: LoadError: InitError: could not load library "/Users/iai/.julia/artifacts/fe31862c531266257045df328d17888573ca2da6/lib/libRmath-julia.dylib"
dlopen(/Users/iai/.julia/artifacts/fe31862c531266257045df328d17888573ca2da6/lib/libRmath-julia.dylib, 0x0001): Library not loaded: '/workspace/destdir/x86_64-apple-darwin14/lib/libgcc_s.1.dylib'
Referenced from: '/Users/iai/.julia/artifacts/fe31862c531266257045df328d17888573ca2da6/lib/libRmath-julia.dylib'
Reason: tried: '/workspace/destdir/x86_64-apple-darwin14/lib/libgcc_s.1.dylib' (no such file), '/Library/Frameworks/R.framework/Resources/lib/libgcc_s.1.dylib' (no such file), '/Library/Java/JavaVirtualMachines/jdk-11.0.18+10/Contents/Home/lib/server/libgcc_s.1.dylib' (no such file)
Stacktrace:
[1] dlopen(s::String, flags::UInt32; throw_error::Bool)
@ Base.Libc.Libdl ./libdl.jl:117
[2] dlopen(s::String, flags::UInt32)
@ Base.Libc.Libdl ./libdl.jl:116
[3] macro expansion
@ ~/.julia/packages/JLLWrappers/pG9bm/src/products/library_generators.jl:63 [inlined]
[4] __init__()
@ Rmath_jll ~/.julia/packages/Rmath_jll/JoEG5/src/wrappers/x86_64-apple-darwin.jl:8
[5] run_module_init(mod::Module, i::Int64)
@ Base ./loading.jl:1128
[6] register_restored_modules(sv::Core.SimpleVector, pkg::Base.PkgId, path::String)
@ Base ./loading.jl:1116
[7] _include_from_serialized(pkg::Base.PkgId, path::String, ocachepath::String, depmods::Vector{Any})
@ Base ./loading.jl:1061
[8] _tryrequire_from_serialized(modkey::Base.PkgId, path::String, ocachepath::String, sourcepath::String, depmods::Vector{Any})
@ Base ./loading.jl:1475
[9] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String, build_id::UInt128)
@ Base ./loading.jl:1568
[10] _require(pkg::Base.PkgId, env::String)
@ Base ./loading.jl:1932
[11] __require_prelocked(uuidkey::Base.PkgId, env::String)
@ Base ./loading.jl:1806
[12] #invoke_in_world#3
@ Base ./essentials.jl:921 [inlined]
[13] invoke_in_world
@ Base ./essentials.jl:918 [inlined]
[14] _require_prelocked(uuidkey::Base.PkgId, env::String)
@ Base ./loading.jl:1797
[15] macro expansion
@ Base ./loading.jl:1784 [inlined]
[16] macro expansion
@ Base ./lock.jl:267 [inlined]
[17] __require(into::Module, mod::Symbol)
@ Base ./loading.jl:1747
[18] #invoke_in_world#3
@ Base ./essentials.jl:921 [inlined]
[19] invoke_in_world
@ Base ./essentials.jl:918 [inlined]
[20] require(into::Module, mod::Symbol)
@ Base ./loading.jl:1740
[21] include
@ Base ./Base.jl:495 [inlined]
[22] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::String)
@ Base ./loading.jl:2216
[23] top-level scope
@ stdin:3
during initialization of module Rmath_jll
in expression starting at /Users/iai/.julia/packages/RCall/dDAVd/src/RCall.jl:1
in expression starting at stdin:3
LoadError("/Library/Frameworks/R.framework/Versions/4.4-x86_64/Resources/library/JuliaCall/julia/setup.jl", 16, ErrorException("Failed to precompile RCall [6f49c342-dc21-5d91-9882-a32aef131414] to \"/Users/iai/.julia/compiled/v1.10/RCall/jl_KF6Z1E\".")) Error in .julia$cmd(paste0(Rhomeset, "Base.include(Main,\"", system.file("julia/setup.jl", :
Error happens when you try to execute command ENV["R_HOME"] = "/Library/Frameworks/R.framework/Resources";Base.include(Main,"/Library/Frameworks/R.framework/Versions/4.4-x86_64/Resources/library/JuliaCall/julia/setup.jl") in Julia.
To have more helpful error messages,
you could considering running the command in Julia directly
>
This is an x86_64 machine running macOS 12.7.5. It is a clean machine with R 4.4.0 from homebrew cask and Julia 1.10.0 from the official installers, starting from an empty Julia depot
R provides libgcc_s.1.1.dylib so the following seemed to fix it:
ln /Library/Frameworks/R.framework/Resources/lib/libgcc_s.1.1.dylib /Library/Frameworks/R.framework/Resources/lib/libgcc_s.1.dylib
but not sure what the right solution here is
@giordano Is this something we should be addressing in our build of Rmath in BB?
I need a full Julia reproducer to have a look, but I'd note that when mixing different runtimes, like Julia's and R's, things get extremely complicated.
Also, I need to know what are the versions of the packages involved, we've been sanitizing the /workspace path for a while now
For what is worth:
(tmp) pkg> st
Status `/private/tmp/Project.toml`
[f50d1b31] Rmath_jll v0.4.2+0
shell> otool -L $(Rmath_jll.libRmath_path)
/Users/mose/.julia/artifacts/fd154813db777405688b4fc611b2b3d6a8c2b820/lib/libRmath-julia.dylib:
@rpath/libRmath-julia.dylib (compatibility version 0.0.0, current version 0.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.60.1)
I don't see libgcc_s at all here and all paths look good to me.
@giordano Here's a complete log of the steps taken, this is an x86_64 machine running macOS 12.7.5. It is clean other than having homebrew installed and has Julia 1.10.4 installed at /Applications/Julia-1.10.4.app/
iai@iais-MacBook-Pro ~ % brew install --cask r
==> Downloading https://cloud.r-project.org/bin/macosx/big-sur-x86_64/base/R-4.4
==> Installing Cask r
==> Running installer for r with sudo; the password may be necessary.
installer: Package name is R 4.4.1 for macOS (X86_64)
installer: Installing at base path /
installer: The install was successful.
🍺 r was successfully installed!
iai@iais-MacBook-Pro ~ % export JULIA_HOME=/Applications/Julia-1.10.4.app/Contents/Resources/julia/bin
iai@iais-MacBook-Pro ~ % R
R version 4.4.1 (2024-06-14) -- "Race for Your Life"
Copyright (C) 2024 The R Foundation for Statistical Computing
Platform: x86_64-apple-darwin20
R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.
Natural language support but running in an English locale
R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.
Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.
> install.packages("JuliaCall")
also installing the dependencies ‘evaluate’, ‘highr’, ‘xfun’, ‘yaml’, ‘Rcpp’, ‘knitr’, ‘rjson’
trying URL 'https://cloud.r-project.org/bin/macosx/big-sur-x86_64/contrib/4.4/evaluate_0.24.0.tgz'
Content type 'application/x-gzip' length 85301 bytes (83 KB)
==================================================
downloaded 83 KB
trying URL 'https://cloud.r-project.org/bin/macosx/big-sur-x86_64/contrib/4.4/highr_0.11.tgz'
Content type 'application/x-gzip' length 37572 bytes (36 KB)
==================================================
downloaded 36 KB
trying URL 'https://cloud.r-project.org/bin/macosx/big-sur-x86_64/contrib/4.4/xfun_0.46.tgz'
Content type 'application/x-gzip' length 529358 bytes (516 KB)
==================================================
downloaded 516 KB
trying URL 'https://cloud.r-project.org/bin/macosx/big-sur-x86_64/contrib/4.4/yaml_2.3.10.tgz'
Content type 'application/x-gzip' length 231527 bytes (226 KB)
==================================================
downloaded 226 KB
trying URL 'https://cloud.r-project.org/bin/macosx/big-sur-x86_64/contrib/4.4/Rcpp_1.0.13.tgz'
Content type 'application/x-gzip' length 3332073 bytes (3.2 MB)
==================================================
downloaded 3.2 MB
trying URL 'https://cloud.r-project.org/bin/macosx/big-sur-x86_64/contrib/4.4/knitr_1.48.tgz'
Content type 'application/x-gzip' length 1165748 bytes (1.1 MB)
==================================================
downloaded 1.1 MB
trying URL 'https://cloud.r-project.org/bin/macosx/big-sur-x86_64/contrib/4.4/rjson_0.2.21.tgz'
Content type 'application/x-gzip' length 214759 bytes (209 KB)
==================================================
downloaded 209 KB
trying URL 'https://cloud.r-project.org/bin/macosx/big-sur-x86_64/contrib/4.4/JuliaCall_0.17.5.tgz'
Content type 'application/x-gzip' length 1116340 bytes (1.1 MB)
==================================================
downloaded 1.1 MB
The downloaded binary packages are in
/var/folders/ll/0556gtxx0s5bcktbyt111cq40000gn/T//RtmpztHbfa/downloaded_packages
> JuliaCall::julia_setup()
Julia version 1.10.4 at location /Applications/Julia-1.10.4.app/Contents/Resources/julia/bin will be used.
Loading setup script for JuliaCall...
ERROR: LoadError: InitError: could not load library "/Users/iai/.julia/artifacts/fe31862c531266257045df328d17888573ca2da6/lib/libRmath-julia.dylib"
dlopen(/Users/iai/.julia/artifacts/fe31862c531266257045df328d17888573ca2da6/lib/libRmath-julia.dylib, 0x0001): Library not loaded: '/workspace/destdir/x86_64-apple-darwin14/lib/libgcc_s.1.dylib'
Referenced from: '/Users/iai/.julia/artifacts/fe31862c531266257045df328d17888573ca2da6/lib/libRmath-julia.dylib'
Reason: tried: '/workspace/destdir/x86_64-apple-darwin14/lib/libgcc_s.1.dylib' (no such file), '/Library/Frameworks/R.framework/Resources/lib/libgcc_s.1.dylib' (no such file), '/Library/Java/JavaVirtualMachines/jdk-11.0.18+10/Contents/Home/lib/server/libgcc_s.1.dylib' (no such file)
Stacktrace:
[1] dlopen(s::String, flags::UInt32; throw_error::Bool)
@ Base.Libc.Libdl ./libdl.jl:117
[2] dlopen(s::String, flags::UInt32)
@ Base.Libc.Libdl ./libdl.jl:116
[3] macro expansion
@ ~/.julia/packages/JLLWrappers/pG9bm/src/products/library_generators.jl:63 [inlined]
[4] __init__()
@ Rmath_jll ~/.julia/packages/Rmath_jll/JoEG5/src/wrappers/x86_64-apple-darwin.jl:8
[5] run_module_init(mod::Module, i::Int64)
@ Base ./loading.jl:1134
[6] register_restored_modules(sv::Core.SimpleVector, pkg::Base.PkgId, path::String)
@ Base ./loading.jl:1122
[7] _include_from_serialized(pkg::Base.PkgId, path::String, ocachepath::String, depmods::Vector{Any})
@ Base ./loading.jl:1067
[8] _tryrequire_from_serialized(modkey::Base.PkgId, path::String, ocachepath::String, sourcepath::String, depmods::Vector{Any})
@ Base ./loading.jl:1481
[9] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String, build_id::UInt128)
@ Base ./loading.jl:1574
[10] _require(pkg::Base.PkgId, env::String)
@ Base ./loading.jl:1938
[11] __require_prelocked(uuidkey::Base.PkgId, env::String)
@ Base ./loading.jl:1812
[12] #invoke_in_world#3
@ ./essentials.jl:926 [inlined]
[13] invoke_in_world
@ ./essentials.jl:923 [inlined]
[14] _require_prelocked(uuidkey::Base.PkgId, env::String)
@ Base ./loading.jl:1803
[15] macro expansion
@ ./loading.jl:1790 [inlined]
[16] macro expansion
@ ./lock.jl:267 [inlined]
[17] __require(into::Module, mod::Symbol)
@ Base ./loading.jl:1753
[18] #invoke_in_world#3
@ ./essentials.jl:926 [inlined]
[19] invoke_in_world
@ ./essentials.jl:923 [inlined]
[20] require(into::Module, mod::Symbol)
@ Base ./loading.jl:1746
[21] include
@ ./Base.jl:495 [inlined]
[22] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::String)
@ Base ./loading.jl:2222
[23] top-level scope
@ stdin:3
during initialization of module Rmath_jll
in expression starting at /Users/iai/.julia/packages/RCall/FEbLj/src/RCall.jl:1
in expression starting at stdin:3
LoadError("/Library/Frameworks/R.framework/Versions/4.4-x86_64/Resources/library/JuliaCall/julia/setup.jl", 16, ErrorException("Failed to precompile RCall [6f49c342-dc21-5d91-9882-a32aef131414] to \"/Users/iai/.julia/compiled/v1.10/RCall/jl_2zxIXD\".")) Error in .julia$cmd(paste0(Rhomeset, "Base.include(Main,\"", system.file("julia/setup.jl", :
Error happens when you try to execute command ENV["R_HOME"] = "/Library/Frameworks/R.framework/Resources";Base.include(Main,"/Library/Frameworks/R.framework/Versions/4.4-x86_64/Resources/library/JuliaCall/julia/setup.jl") in Julia.
To have more helpful error messages,
you could considering running the command in Julia directly
>
Save workspace image? [y/n/c]: n
Julia exit.
iai@iais-MacBook-Pro ~ % /Applications/Julia-1.10.4.app/Contents/Resources/julia/bin/julia
_
_ _ _(_)_ | Documentation: https://docs.julialang.org
(_) | (_) (_) |
_ _ _| |_ __ _ | Type "?" for help, "]?" for Pkg help.
| | | | | | |/ _` | |
| | |_| | | | (_| | | Version 1.10.4 (2024-06-04)
_/ |\__'_|_|_|\__'_| | Official https://julialang.org/ release
|__/ |
(@v1.10) pkg> st
Status `~/.julia/environments/v1.10/Project.toml`
[6f49c342] RCall v0.14.4
[fd094767] Suppressor v0.2.7
(@v1.10) pkg> st -m
Status `~/.julia/environments/v1.10/Manifest.toml`
[49dc2e85] Calculus v0.5.1
[324d7699] CategoricalArrays v0.10.8
[34da2185] Compat v4.16.0
[8f4d0f93] Conda v1.10.2
[a8cc5b0e] Crayons v4.1.1
[9a962f9c] DataAPI v1.16.0
[a93c6f00] DataFrames v1.6.1
[864edb3b] DataStructures v0.18.20
[e2d170a0] DataValueInterfaces v1.0.0
[ffbed154] DocStringExtensions v0.9.3
[fa6b7ba4] DualNumbers v0.6.8
[34004b35] HypergeometricFunctions v0.3.23
[842dd82b] InlineStrings v1.4.2
[41ab1584] InvertedIndices v1.3.0
[92d709cd] IrrationalConstants v0.2.2
[82899510] IteratorInterfaceExtensions v1.0.0
[692b3bcd] JLLWrappers v1.5.0
[682c06a0] JSON v0.21.4
[b964fa9f] LaTeXStrings v1.3.1
[2ab3a3ac] LogExpFunctions v0.3.28
[e1d29d7a] Missings v1.2.0
[77ba4419] NaNMath v1.0.2
[bac558e1] OrderedCollections v1.6.3
[69de0a69] Parsers v2.8.1
[2dfb63ee] PooledArrays v1.4.3
[aea7be01] PrecompileTools v1.2.1
[21216c6a] Preferences v1.4.3
[08abe8d2] PrettyTables v2.3.2
[6f49c342] RCall v0.14.4
[189a3867] Reexport v1.2.2
[ae029012] Requires v1.3.0
[79098fc4] Rmath v0.7.1
[91c51154] SentinelArrays v1.4.5
[1277b4bf] ShiftedArrays v2.0.0
[a2af1166] SortingAlgorithms v1.2.1
[276daf66] SpecialFunctions v2.4.0
[82ae8749] StatsAPI v1.7.0
[2913bbd2] StatsBase v0.34.3
[4c63d2b9] StatsFuns v1.3.1
[3eaba693] StatsModels v0.7.3
[892a3eda] StringManipulation v0.3.4
[fd094767] Suppressor v0.2.7
[3783bdb8] TableTraits v1.0.1
[bd369af6] Tables v1.12.0
[81def892] VersionParsing v1.3.0
[1b915085] WinReg v1.0.0
[efe28fd5] OpenSpecFun_jll v0.5.5+0
[f50d1b31] Rmath_jll v0.4.2+0
[0dad84c5] ArgTools v1.1.1
[56f22d72] Artifacts
[2a0f44e3] Base64
[ade2ca70] Dates
[f43a241f] Downloads v1.6.0
[7b1f6079] FileWatching
[9fa8497b] Future
[b77e0a4c] InteractiveUtils
[b27032c2] LibCURL v0.6.4
[76f85450] LibGit2
[8f399da3] Libdl
[37e2e46d] LinearAlgebra
[56ddb016] Logging
[d6f4376e] Markdown
[a63ad114] Mmap
[ca575930] NetworkOptions v1.2.0
[44cfe95a] Pkg v1.10.0
[de0858da] Printf
[3fa0cd96] REPL
[9a3f8284] Random
[ea8e919c] SHA v0.7.0
[9e88b42a] Serialization
[6462fe0b] Sockets
[2f01184e] SparseArrays v1.10.0
[10745b16] Statistics v1.10.0
[fa267f1f] TOML v1.0.3
[a4e569a6] Tar v1.10.0
[cf7118a7] UUIDs
[4ec0a83e] Unicode
[e66e0078] CompilerSupportLibraries_jll v1.1.1+0
[deac9b47] LibCURL_jll v8.4.0+0
[e37daf67] LibGit2_jll v1.6.4+0
[29816b5a] LibSSH2_jll v1.11.0+1
[c8ffd9c3] MbedTLS_jll v2.28.2+1
[14a3606d] MozillaCACerts_jll v2023.1.10
[4536629a] OpenBLAS_jll v0.3.23+4
[05823500] OpenLibm_jll v0.8.1+2
[bea87d4a] SuiteSparse_jll v7.2.1+1
[83775a58] Zlib_jll v1.2.13+1
[8e850b90] libblastrampoline_jll v5.8.0+1
[8e850ede] nghttp2_jll v1.52.0+1
[3f19e933] p7zip_jll v17.4.0+2
julia> using Rmath_jll
│ Package Rmath_jll not found, but a package named Rmath_jll is available from
│ a registry.
│ Install package?
│ (@v1.10) pkg> add Rmath_jll
└ (y/n/o) [y]:
Updating registry at `~/.julia/registries/General.toml`
Resolving package versions...
Updating `~/.julia/environments/v1.10/Project.toml`
[f50d1b31] + Rmath_jll v0.4.2+0
No Changes to `~/.julia/environments/v1.10/Manifest.toml`
shell> otool -L $(Rmath_jll.libRmath_path)
/Users/iai/.julia/artifacts/fe31862c531266257045df328d17888573ca2da6/lib/libRmath-julia.dylib:
@rpath/libRmath-julia.dylib (compatibility version 0.0.0, current version 0.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1238.0.0)
/workspace/destdir/x86_64-apple-darwin14/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)
Please let me know if there's anything else you want me to run here
Ok, so, that's specific to the x86_64 build, not the aarch64 one (which is what I was looking at above). But we normalize rpaths in https://github.com/JuliaPackaging/BinaryBuilder.jl/blob/24a227b48f8db91bb03a17265648069897a1c050/src/auditor/dynamic_linkage.jl, and we even have tests (including x86_64-darwin): https://github.com/JuliaPackaging/BinaryBuilder.jl/blob/24a227b48f8db91bb03a17265648069897a1c050/test/auditing.jl#L602-L647. Someone would have to dig down what's going wrong, but I don't have much time myself
@giordano Just out of curiosity, is there anything we can do to the Rmath build to force it to not hardcode the full path for libgcc?
I guess one thing to do is understand why it's linking to libgcc_s in the first place: it looks wrong, and it's happening only on the x86_64 build, not the aarch64 one
For example https://github.com/JuliaStats/Rmath-julia/blob/18dcd7c259b031c3ce9c275b7dd136585d126017/Make.inc#L16-L17 https://github.com/JuliaStats/Rmath-julia/blob/18dcd7c259b031c3ce9c275b7dd136585d126017/Make.inc#L21-L28 looks unnecessary, and wrong
I can't close issues in this repository, but this is fixed by https://github.com/JuliaPackaging/Yggdrasil/pull/9228. I opened #47 to explain what's wrong with the build system of this package.
Folks, please try Pkg.update() and let us know if there's an issue. I tried it on my x64 mac and it works fine.