Qt library conflict with v0.57.3

[jjstickel]

With GR v0.57.3 installed, I get the following when using Plots with the pyplot() backend:

julia> using Plots

julia> pyplot()
objc[3586]: Class QMacAutoReleasePoolTracker is implemented in both /Users/jstickel/.julia/artifacts/ee74d003b12e6dd5f748e8cefb1688edbdfb184b/lib/QtCore.framework/Versions/5/QtCore (0x16e94c9b8) and /opt/local/libexec/qt5/lib/QtCore.framework/Versions/5/QtCore (0x1731d5448). One of the two will be used. Which one is undefined.
objc[3586]: Class QT_ROOT_LEVEL_POOL__THESE_OBJECTS_WILL_BE_RELEASED_WHEN_QAPP_GOES_OUT_OF_SCOPE is implemented in both /Users/jstickel/.julia/artifacts/ee74d003b12e6dd5f748e8cefb1688edbdfb184b/lib/QtCore.framework/Versions/5/QtCore (0x16e94ca30) and /opt/local/libexec/qt5/lib/QtCore.framework/Versions/5/QtCore (0x1731d54c0). One of the two will be used. Which one is undefined.
objc[3586]: Class KeyValueObserver is implemented in both /Users/jstickel/.julia/artifacts/ee74d003b12e6dd5f748e8cefb1688edbdfb184b/lib/QtCore.framework/Versions/5/QtCore (0x16e94ca58) and /opt/local/libexec/qt5/lib/QtCore.framework/Versions/5/QtCore (0x1731d54e8). One of the two will be used. Which one is undefined.
objc[3586]: Class RunLoopModeTracker is implemented in both /Users/jstickel/.julia/artifacts/ee74d003b12e6dd5f748e8cefb1688edbdfb184b/lib/QtCore.framework/Versions/5/QtCore (0x16e94caa8) and /opt/local/libexec/qt5/lib/QtCore.framework/Versions/5/QtCore (0x1731d5538). One of the two will be used. Which one is undefined.
Plots.PyPlotBackend()

Subsequently running plot() causes a segfault. This does not happen if I set the GR version to 0.55.0. I am using MacPorts-installed python (ENV["PYTHON"] = "python3.8"), but I also noticed a Qt library conflict with a Qt library in ~/.julia/conda/ when I temporarily tried using that version of python (ENV["PYTHON"] = ""). I couldn't actually make that python work due to a separate problem, though. In any case, I should be able to use a system-installed python.

[mkitti]

Do you still get the error if you set the binary provider to "GR"?

julia> ENV["JULIA_GR_PROVIDER"] = "GR" # Do this in a new Julia session
"GR"

julia> import Pkg; Pkg.build("GR")
    Building GR → `C:\Users\kittisopikulm\.julia\dev\GR\deps\build.log`

julia> using GR

julia> GR.gr_provider # Confirm that the provider is set to GR
Base.RefValue{String}("GR")

The more recent versions of GR make use of GR_jll by default which in turn makes of Qt5Base_jll

Using the above steps you can switch it to use tarballs built outside of BinaryBuilder.jl.

[jheinen]

I had the same problem today (at our site) and the only solution I found was to pin jlqml_jll to v0.1.5. We use the original Qt 5.15.2 distribution (from qt.io, for Julia, Python and C++) and a locally installed version of jlqml and libcxxwrap-julia (defined using JLQML_DIR and JLCXX_DIR) for Julia. This did no longer work after upgrading the libraries and switching to v0.2.0.

[jjstickel]

Do you still get the error if you set the binary provider to "GR"?

julia> ENV["JULIA_GR_PROVIDER"] = "GR" # Do this in a new Julia session
"GR"

julia> import Pkg; Pkg.build("GR")

Yes, this seems to have resolved the problem I had. Make this default behavior somehow? I even wiped my .julia folder at one point and still experienced the problem, so it wasn't due to old settings.

[mkitti]

GR_jll and the BinaryBuilder ecosystem make sense as a default when using a pure Julia environment. It gets complicated when you start adding Python or need to use the same library outside of Julia.

However, it does not seem right that we load the GR shared libraries eagerly when your intent is to use PyPlot. I've been thinking of replacing Libdl.dlsym with something that caches. We could also make loading the GR libraries lazy such that they only load when someone actually tries to plot with GR.

[mkitti]

I had the same problem today (at our site) and the only solution I found was to pin jlqml_jll to v0.1.5

Would it make sense to pin down package dependencies in build_tarballs.jl ?

https://github.com/JuliaPackaging/BinaryBuilder.jl/blob/master/docs/src/building.md#version-number-of-dependencies

[jheinen]

I still don't see why it worked before. I asked @barche for help - may be, he has an idea.

[jjstickel]

However, it does not seem right that we load the GR shared libraries eagerly when your intent is to use PyPlot. I've been thinking of replacing Libdl.dlsym with something that caches. We could also make loading the GR libraries lazy such that they only load when someone actually tries to plot with GR.

Right, I can't imagine a scenario where I (or most anyone) would want to use PyPlot and GR in the same REPL session. Lazy loading seems like a reasonable approach.

[barche]

This error occurs when multiple installations of Qt5 get loaded at the same time. The /Users/jstickel/.julia/artifacts/ee74d003b12e6dd5f748e8cefb1688edbdfb184b/lib/QtCore.framework/Versions/5/QtCore is the library from Qt5Base_jll, which gets pulled in by the latest GR.

There is indeed a conflict with QML.jl right now, because the last released QML.jl still depends on Qt_jll, which conflicts with Qt5Base_jll, but I don't think that is the issue here. Something is loading Qt from /opt/local/libexec/qt5, but I don't know what.

The most elegant workaround I can think of is to make a ~/.julia/artifacts/Overrides.toml with the following contents:

[ea2cea3b-5b76-57ae-a6ef-0a8af62496e1]
Qt5Base = "/opt/local/libexec/qt5"

[jheinen]

Thanks @barche . I could have have thought of it myself ...

[mkitti]

This looks related: https://discourse.julialang.org/t/cant-install-plots-on-julia-1-6-0/59360/9

julia> import Pkg; Pkg.add("GR_jll"); import GR_jll
    Updating registry at `~/.julia/registries/General`
   Resolving package versions...
    Updating `~/.julia/environments/v1.6/Project.toml`
  [d2c73de3] + GR_jll v0.57.2+0
  No Changes to `~/.julia/environments/v1.6/Manifest.toml`
[ Info: Precompiling GR_jll [d2c73de3-f751-5644-a686-071e5b155ba9]
ERROR: LoadError: LoadError: InitError: could not load library "/home/gibson/.julia/artifacts/7835210e9722f2d04d4e74972925b5c43d9cfcaa/lib/libQt5Concurrent.so"
/usr/lib64/libQt5Core.so.5: version `Qt_5.15' not found (required by /home/gibson/.julia/artifacts/7835210e9722f2d04d4e74972925b5c43d9cfcaa/lib/libQt5Concurrent.so)
Stacktrace:
  [1] dlopen(s::String, flags::UInt32; throw_error::Bool)
    @ Base.Libc.Libdl ./libdl.jl:114
  [2] dlopen(s::String, flags::UInt32)
    @ Base.Libc.Libdl ./libdl.jl:114
  [3] macro expansion
    @ ~/.julia/packages/JLLWrappers/WnWcZ/src/products/library_generators.jl:63 [inlined]
  [4] __init__()
    @ Qt5Base_jll ~/.julia/packages/Qt5Base_jll/Ks8eF/src/wrappers/x86_64-linux-gnu-cxx11.jl:31
  [5] _include_from_serialized(path::String, depmods::Vector{Any})
    @ Base ./loading.jl:674
  [6] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String)
    @ Base ./loading.jl:760
  [7] _require(pkg::Base.PkgId)
    @ Base ./loading.jl:998
  [8] require(uuidkey::Base.PkgId)
    @ Base ./loading.jl:914
  [9] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:901
 [10] include(mod::Module, _path::String)
    @ Base ./Base.jl:386
 [11] top-level scope
    @ ~/.julia/packages/JLLWrappers/WnWcZ/src/toplevel_generators.jl:170
 [12] include
    @ ./Base.jl:386 [inlined]
 [13] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::Nothing)
    @ Base ./loading.jl:1213
 [14] top-level scope
    @ none:1
 [15] eval
    @ ./boot.jl:360 [inlined]
 [16] eval(x::Expr)
    @ Base.MainInclude ./client.jl:446
 [17] top-level scope
    @ none:1
during initialization of module Qt5Base_jll
in expression starting at /home/gibson/.julia/packages/GR_jll/lXOy4/src/wrappers/x86_64-linux-gnu-cxx11.jl:13
in expression starting at /home/gibson/.julia/packages/GR_jll/lXOy4/src/GR_jll.jl:2
ERROR: Failed to precompile GR_jll [d2c73de3-f751-5644-a686-071e5b155ba9] to /home/gibson/.julia/compiled/v1.6/GR_jll/jl_JK86kc.
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:33
 [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::Base.TTY, internal_stdout::Base.TTY)
   @ Base ./loading.jl:1360
 [3] compilecache(pkg::Base.PkgId, path::String)
   @ Base ./loading.jl:1306
 [4] _require(pkg::Base.PkgId)
   @ Base ./loading.jl:1021
 [5] require(uuidkey::Base.PkgId)
   @ Base ./loading.jl:914
 [6] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:901

[lewisl]

Not sure how viable a solution this is.

1. Requires a bit of a hack to include Overrides.toml file when installing Julia.
2. Sort of has an artifact problem as below. Note this IS a self-healing problem because GR fixes itself. Just a shame it happens at all.

julia> plot(1:5)
┌ Error: Error importing GR_jll:
│   err =
│    InitError: Artifact "Qt5Base" was not installed correctly. Try `using Pkg; Pkg.instantiate()` to re-install all missing resources.
│    during initialization of module Qt5Base_jll
└ @ GR ~/.julia/packages/GR/Pwj8N/src/funcptrs.jl:42
Your GR installation is incomplete. Rerunning build step for GR package.
┌ Info: Switching provider to GR due to error in depsfile
└   depsfile = "/Users/lewislevin/.julia/packages/GR/Pwj8N/deps/deps.jl"
┌ Info: Emptying depsfile. GR provider is GR
│   provider = "GR"
└   depsfile = "/Users/lewislevin/.julia/packages/GR/Pwj8N/deps/deps.jl"
[ Info: Downloading pre-compiled GR 0.61.0 Darwin binary
[ Info: GR was successfully rebuilt

[jjstickel]

So, it's nearly 3 years later, and this problem still exists. But now, AFAICT, JULIA_GR_PROVIDER is no longer used by GR. I was able to workaround it via ~/.julia/artifacts/Overrides.toml:

b37aebc8943f796682edea53e2ec10463bf624ac = "/opt/local/libexec/qt5/"

But only after I restricted GR to <0.72.9, which still uses Qt5 rather than Qt6 (macports doesn't yet have pyqt6). Any progress with the lazy loading concept? Maybe using package extensions? Would that be an issue for Plots.jl?

[jheinen]

Unfortunately, I don't see any way of solving this problem at the GR level. Both the GKS QtTerm and GRplot are independent and self-contained binaries. In addition, we provide our "own" distributions for various operating systems and for different packaging environments.

The problem arose from the need to precompile everything (to improve the TTFP with Plots.jl), which is specific to Julia. This is why we still prefer system-wide GR distributions on our systems.