Julia loading Julia's libstdc++ instead of system, causes inscrutable errors

[weech]

Hi, GLFW.jl is affected by the outdated libstdc++ issue. It's a problem that has been affecting several projects and has been documented in at least one Discourse thread. What happens is that Julia ships with an old version of libstdc++, and something in the linking process fails on systems that expect a newer version of libstdc++. If you run a GLFW.jl function with the LIBGL_DEBUG=verbose environment variable set, you get an error message like libGL: MESA-LOADER: failed to open /usr/lib/dri/radeonsi_dri.so: /home/alex/pkgs/julia-1.3.0/bin/../lib/julia/libstdc++.so.6: version GLIBCXX_3.4.26' not found (required by /usr/lib/libLLVM-9.so)`. The libstdc++ shipped with Julia 1.3 is version 3.4.24.

There's an open issue on the main Julia repo. Other packages like RCall have been adding notices in their docs with workarounds in the meantime. The error message GLFW.jl gives right now is especially lacking. Without the LIGBL_DEBUG=verbose, on my machine it is:

libGL error: MESA-LOADER: failed to open radeonsi (search paths /usr/lib/dri)
libGL error: failed to load driver: radeonsi
libGL error: MESA-LOADER: failed to open radeonsi (search paths /usr/lib/dri)
libGL error: failed to load driver: radeonsi
libGL error: MESA-LOADER: failed to open swrast (search paths /usr/lib/dri)
libGL error: failed to load driver: swrast
ERROR: GLFWError (VERSION_UNAVAILABLE): GLX: Failed to create context: BadValue
Stacktrace:
 [1] _ErrorCallbackWrapper(::Int32, ::Cstring) at /home/alex/.julia/packages/GLFW/g1nX6/src/callback.jl:43
 [2] CreateWindow(::Int64, ::Int64, ::String, ::GLFW.Monitor, ::GLFW.Window) at /home/alex/.julia/packages/GLFW/g1nX6/src/glfw3.jl:487 (repeats 2 times)
 [3] top-level scope at REPL[3]:1

The workaround right now is to:

1. Delete julia-1.3.0/lib/julia/libstdc++.so.6
2. Find the system libstdc++ with whereis libstdc++
3. Link that location to julia-1.3.0/lib/julia-libstdc++.so.6 using ln.

I think it'd be worthwhile to put a notice up on the README that this issue is occurring. I encountered the problem while trying to use Makie, and this repo was one of the first places I went to debug it since the error comes from this library. It's probably worthwhile to put a notice up in Makie too.

[giordano]

What version of GLFW.jl are you using?

[weech]

The Manifest lists GLFW_jll version = "3.3.2+0" and GLFW version = "3.2.2". The GLFW from the repos is "glfw-x11 3.3.2-1".

[SimonDanisch]

Is there any way we can fix this? Seems like quite a few people run into this!

[giordano]

There is this issue: https://github.com/JuliaLang/julia/issues/34276

[giordano]

Is there a MWE to reproduce the error? I have GCC v9.3.0 with libstdc++.so.6.0.28 on my system and the example in the README.md works for me.

[SimonDanisch]

I think you need an AMD card, since this seems to be a radeon driver issue

[weech]

Yes, I have an AMD card, and it is the radeon driver specifically that can't be loaded while Julia's included libstdc++ is loaded.

[dawbarton]

It's not just radeon; I've just been hit by this and I've got an Intel integrated GPU with the iris driver. Thankfully now works with this fix :smile:

[Kevin-Mattheus-Moerman]

I have this issue for testing Makie (Ubuntu 20.04, Julia 1.4, NVIDIA

@weech thanks for posting the work around. I'm new to Julia. Can you please elaborate on your work around.

1. Delete julia-1.3.0/lib/julia/libstdc++.so.6 this step is clear
2. Find the system libstdc++ with whereis libstdc++ This just returns: libstdc++:, am I missing something?
3. Link that location to julia-1.3.0/lib/julia-libstdc++.so.6 using ln. How does this linking work can you provide me with some information?

Thanks.

[weech]

Hi, After more experimentation, it turns out steps 2 and 3 shouldn't be necessary (at least it works on my machine 😉). Just removing the libstdc++ included with Julia should be enough to stop the linker errors. (Julia will load the system libstdc++ automatically if not provided an alternative).

[Kevin-Mattheus-Moerman]

@weech thanks for the quick reply. I tried doing just step 1 but unfortunately it still does not work. When I try ] build GLFW in Julia I get nothing, not even a warning message, an error or anything, just an empty line. I then tried ] build GLMakie and got:

Building ModernGL → `~/.julia/packages/ModernGL/rVuW2/deps/build.log`
Building GLMakie ─→ `~/.julia/packages/GLMakie/2xw8L/deps/build.log`
Error: Error building `GLMakie`: 
│ libGL error: No matching fbConfigs or visuals found
│ libGL error: failed to load driver: swrast
│ init error of GLFW
│ ERROR: LoadError: OpenGL/GLFW wasn't installed correctly. This likely means,
│ you don't have an OpenGL capable Graphic Card,
│ you don't have the newest video driver installed,
│ or the GLFW build failed. If you're on linux and `]build` GLFW failed,
│ try manually adding `sudo apt-get install libglfw3` and then `]build GLMakie`.
...
...

FYI when I run glxinfo | grep "OpenGL version" in a terminal I get:

OpenGL version string: 4.4.0 NVIDIA 340.108

Any suggestions?

Thanks. (fyi @SimonDanisch, still stuck trying to set-up Makie)

[weech]

I think this might be a separate problem? I haven't seen the "No matching fbConfigs or visuals found" error before, and there's no error saying it couldn't load your driver. Can you compile and run the following program to test that the system GLFW is working? It's just a hello world app from the GLFW website that creates a window with the title "Hello World". Since you're on Ubuntu you might have to install the dev libraries and tweak where the headers are. I compiled it with clang $(pkg-config --cflags glfw3) -o glfwtest glfwtest.c $(pkg-config --libs glfw3) -lGL. Another debugging step would be to run Julia with the environment variable LIBGL_DEBUG=verbose so you get additional information.

#include <GL/gl.h>
#include <GLFW/glfw3.h>

int main(void)
{
    GLFWwindow* window;

    /* Initialize the library */
    if (!glfwInit())
        return -1;

    /* Create a windowed mode window and its OpenGL context */
    window = glfwCreateWindow(640, 480, "Hello World", NULL, NULL);
    if (!window)
    {
        glfwTerminate();
        return -1;
    }

    /* Make the window's context current */
    glfwMakeContextCurrent(window);

    /* Loop until the user closes the window */
    while (!glfwWindowShouldClose(window))
    {
        /* Render here */
        glClear(GL_COLOR_BUFFER_BIT);

        /* Swap front and back buffers */
        glfwSwapBuffers(window);

        /* Poll for and process events */
        glfwPollEvents();
    }

    glfwTerminate();
    return 0;
}

[Kevin-Mattheus-Moerman]

@weech thanks

I think it worked, if the picture below is correct.

I followed these steps:

1. I created that file and called it glfwtest.c
2. I then did: sudo apt install clang
3. I then ran clang $(pkg-config --cflags glfw3) -o glfwtest glfwtest.c $(pkg-config --libs glfw3) -lGL
4. I then noticed the executable in the folder, if I run that I get:

How do I run julia with that environment variable LIBGL_DEBUG=verbose? I went here: https://docs.julialang.org/en/v1/manual/environment-variables/index.html I'll try ENV["LIBGL_DEBUG"] = "verbose" based on that link and will report back.

[Kevin-Mattheus-Moerman]

After running ENV["LIBGL_DEBUG"] = "verbose" in the REPL I tried ] build Makie again and got:

   Building ModernGL → `~/.julia/packages/ModernGL/rVuW2/deps/build.log`
   Building GLMakie ─→ `~/.julia/packages/GLMakie/S9Zib/deps/build.log`
┌┌ Error: Error building `GLMakie`: 
│ libGL: screen 0 does not appear to be DRI2 capable
│ libGL: Can't open configuration file /etc/drirc: No such file or directory.
│ libGL: Can't open configuration file /home/kevin/.drirc: No such file or directory.
│ libGL: Can't open configuration file /etc/drirc: No such file or directory.
│ libGL: Can't open configuration file /home/kevin/.drirc: No such file or directory.
│ libGL error: No matching fbConfigs or visuals found
│ libGL error: failed to load driver: swrast
│ init error of GLFW
│ ERROR: LoadError: OpenGL/GLFW wasn't installed correctly. This likely means,
│ you don't have an OpenGL capable Graphic Card,
│ you don't have the newest video driver installed,
│ or the GLFW build failed. If you're on linux and `]build` GLFW failed,
│ try manually adding `sudo apt-get install libglfw3` and then `]build GLMakie`.
│ If you're on a headless server, you still need to install x-server and
│ proper GPU drivers. You can take inspiration from this article
│ on how to get Makie running on a headless system:
│ https://nextjournal.com/sdanisch/makie-1.0
│ If you don't have a GPU, there is also a Cairo software backend
│ for Makie which you can use:
│ https://github.com/JuliaPlots/CairoMakie.jl.
│ Please check the below error and open an issue at:
│ https://github.com/JuliaPlots/GLMakie.jl.
│ After you fixed your OpenGL install, please run `]build GLMakie` again!
│ GLMakie will still load, but will be disabled as a default backend for Makie
│ 
│ Stacktrace:
│  [1] error(::String) at ./error.jl:33
│  [2] top-level scope at /home/kevin/.julia/packages/GLMakie/S9Zib/deps/build.jl:63
│  [3] include(::String) at ./client.jl:439
│  [4] top-level scope at none:5
│ in expression starting at /home/kevin/.julia/packages/GLMakie/S9Zib/deps/build.jl:31
│ caused by [exception 1]
│ GLFWError (VERSION_UNAVAILABLE): GLX: Failed to create context: BadMatch
│ Stacktrace:
│  [1] _ErrorCallbackWrapper(::Int32, ::Cstring) at /home/kevin/.julia/packages/GLFW/g1nX6/src/callback.jl:43
│  [2] CreateWindow(::Int64, ::Int64, ::String, ::GLFW.Monitor, ::GLFW.Window) at /home/kevin/.julia/packages/GLFW/g1nX6/src/glfw3.jl:487
│  [3] GLFW.Window(; name::String, resolution::Tuple{Int64,Int64}, debugging::Bool, major::Int64, minor::Int64, windowhints::Array{Tuple{UInt32,Int64},1}, contexthints::Array{Tuple{UInt32,Integer},1}, visible::Bool, focus::Bool, fullscreen::Bool, monitor::Nothing, share::GLFW.Window) at /home/kevin/.julia/packages/GLFW/g1nX6/src/glfw3.jl:338
│  [4] top-level scope at /home/kevin/.julia/packages/GLMakie/S9Zib/deps/build.jl:34
│  [5] include(::String) at ./client.jl:439
│  [6] top-level scope at none:5
└ @ Pkg.Operations /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Pkg/src/Operations.jl:892
   Building FFTW ────→ `~/.julia/packages/FFTW/5DZuu/deps/build.log`

I still get nothing after running build GLFW.

[Kevin-Mattheus-Moerman]

I upgraded to nvidia-driver-390 (was using 340) and now Makie does build without errors. So that resolved my issue. Thanks for your help @weech.

[mateuszatki]

I got

  Building FFTW ────→ `~/.julia/packages/FFTW/DMUbN/deps/build.log`
   Building GLMakie ─→ `~/.julia/packages/GLMakie/4EXKe/deps/build.log`
┌ Error: Error building `GLMakie`: 
│ libGL error: MESA-LOADER: failed to open iris (search paths /usr/lib/x86_64-linux-gnu/dri:\$${ORIGIN}/dri:/usr/lib/dri)
│ libGL error: failed to load driver: iris
│ libGL error: MESA-LOADER: failed to open iris (search paths /usr/lib/x86_64-linux-gnu/dri:\$${ORIGIN}/dri:/usr/lib/dri)
│ libGL error: failed to load driver: iris
│ libGL error: MESA-LOADER: failed to open swrast (search paths /usr/lib/x86_64-linux-gnu/dri:\$${ORIGIN}/dri:/usr/lib/dri)
│ libGL error: failed to load driver: swrast
│ init error of GLFW

Unfortunately proposed walk-around sounds a bit hacky, as in my opinion replacing stdc++ with other version for which Julia's binaries weren't build although might solve GLMakie build problem, it may as well cause other issues.

What worked for me instead (Nvidia users only) I just forced julia to use Nvidia with offloading in latest drivers http://us.download.nvidia.com/XFree86/Linux-x86_64/450.66/README/primerenderoffload.html

__NV_PRIME_RENDER_OFFLOAD=1 __GLX_VENDOR_LIBRARY_NAME=nvidia julia

[SimonDanisch]

@jayschwa, you think we could improve the situation, by loading the systems glfw library? I think this is a BinaryBuilder problem (well not alone, but I think the systems glfw doesn't have this problem), so something like this may just work:

try 
  using GLFW_jll
catch e
  @warn "BinaryBuilder GLFW doesn't load see issue #198, trying to load system GLFW" exception=e
  try
    const libglfw = Libdl.open("...") # I kinda forgot how to do this cleanly, but shouldn't be to hard to figure out
  catch e
    @error "Could not load system GLFW, make sure its installed (show some tips)" exception=e
  end
end

Anyone running into this issue care to try out such a solution?

[weech]

I gave roughly that a try, and it didn't help anything. I replaced using GLFW_jll with

global libglfw_handle = Libdl.dlopen("/usr/lib/libglfw.so") 
const libglfw = "libglfw"

and ran the tests, and it gave the normal GPU driver errors. Again, the problem is that Julia ships a version of libstdc++ that is too old for the GPU drivers and that outdated library is loaded when Julia is launched. For this problem to go away, either the GPU drivers need to link against an older libstdc++ (i.e., build them all in BinaryBuilder) or Julia needs to ship a newer one.

By the way, this issue was opened in 1.3 times, but it's still a problem with 1.5.1.

[SimonDanisch]

Ok, so bumping this issue seems to be our only hope: https://github.com/JuliaLang/julia/issues/37200 https://github.com/JuliaLang/julia/issues/34276

[giordano]

An alternative is building Julia from source, that should use system libstdc++ as far as I know :upside_down_face:

[sylvaticus]

Just removing julia-1.5.0/lib/julia/libstdc++.so.6 worked for me (Ubuntu 20.04 on a Dell Laptop with Intel graphics) but I hope it will not create problems in other areas..

[mashu]

An alternative is building Julia from source, that should use system libstdc++ as far as I know upside_down_face

I should justify why I was not going system's stdc route. In the past I experienced Julia crashes on not very obvious but basic operations because I was using Debian's package and not official binaries. I opened these as issues and I was told to switch to more stable official binaries which solved the problem. Of course these bugs should have been fixed, but linking against something unofficial is stepping into very untested area.

[j-fu]

Just removing julia-1.5.0/lib/julia/libstdc++.so.6 worked for me (Ubuntu 20.04 on a Dell Laptop with Intel graphics) but I hope it will not create problems in other areas..

You also can do (in bash)

$ export LD_PRELOAD=/usr/lib64/libstdc++.so.6
$ julia

I had to do this for a similar reason and put it into my julia start script... Though my GLFW works without.

[SimonDanisch]

Btw, the branched off 1.6 finally seems to fix this :)

[FeldrinH]

1.6.1 is now released. Is the issue resolved on that?

[SimonDanisch]

Somewhat ;) Should mostly work now, but I think the problem is, that it might come up again, whenever things get out of sync

[koehlerson]

I have the same situation as @j-fu on my laptop. Used to work without preloading the stdc++ lib

[blegat]

Somewhat ;) Should mostly work now, but I think the problem is, that it might come up again, whenever things get out of sync

On ArchLinux with an integrated Intel GPU with mesa driver, I get:

$ glxinfo
OpenGL vendor string: Intel
OpenGL renderer string: Mesa Intel(R) HD Graphics 530 (SKL GT2)
OpenGL core profile version string: 4.6 (Core Profile) Mesa 21.1.4

But trying to use GLMakie, it is expecting GLIBCXX_3.4.29 so I get the error:

libGL error: MESA-LOADER: failed to open iris: /opt/julias/julia-1.6.2/bin/../lib/julia/libstdc++.so.6: version `GLIBCXX_3.4.29' not found (required by /usr/lib/dri/iris_dri.so) (search paths /usr/lib/dri)
libGL error: failed to load driver: iris
libGL error: MESA-LOADER: failed to open iris: /opt/julias/julia-1.6.2/bin/../lib/julia/libstdc++.so.6: version `GLIBCXX_3.4.29' not found (required by /usr/lib/dri/iris_dri.so) (search paths /usr/lib/dri)
libGL error: failed to load driver: iris
libGL error: MESA-LOADER: failed to open swrast: /opt/julias/julia-1.6.2/bin/../lib/julia/libstdc++.so.6: version `GLIBCXX_3.4.29' not found (required by /usr/lib/dri/swrast_dri.so) (search paths /usr/lib/dri)
libGL error: failed to load driver: swrast
┌ Warning:     GLFW couldn't create an OpenGL window.
│     This likely means, you don't have an OpenGL capable Graphic Card,
│     or you don't have an OpenGL 3.3 capable video driver installed.
│     Have a look at the troubleshooting section in the GLMakie readme:
│     https://github.com/JuliaPlots/Makie.jl/tree/master/GLMakie#troubleshooting-opengl.
└ @ GLMakie ~/.julia/packages/GLMakie/rXGL8/src/screen.jl:336
Error showing value of type Makie.FigureAxisPlot:
ERROR: GLFWError (VERSION_UNAVAILABLE): GLX: Failed to create context: GLXBadFBConfig
Stacktrace:
  [1] _ErrorCallbackWrapper(code::Int32, description::Cstring)
    @ GLFW ~/.julia/packages/GLFW/BWxfF/src/callback.jl:43
  [2] CreateWindow(width::Int64, height::Int64, title::String, monitor::GLFW.Monitor, share::GLFW.Window)
    @ GLFW ~/.julia/packages/GLFW/BWxfF/src/glfw3.jl:499
  [3] GLFW.Window(; name::String, resolution::Tuple{Int64, Int64}, debugging::Bool, major::Int64, minor::Int64, windowhints::Vector{Tuple{UInt32, Integer}}, contexthints::Vector{Tuple{UInt32, Integer}}, visible::Bool, focus::Bool, fullscreen::Bool, monitor::Nothing, share::GLFW.Window)
    @ GLFW ~/.julia/packages/GLFW/BWxfF/src/glfw3.jl:344
  [4] GLMakie.Screen(; resolution::Tuple{Int64, Int64}, visible::Bool, title::String, kw_args::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ GLMakie ~/.julia/packages/GLMakie/rXGL8/src/screen.jl:328
  [5] Screen
    @ ~/.julia/packages/GLMakie/rXGL8/src/screen.jl:301 [inlined]
  [6] global_gl_screen
    @ ~/.julia/packages/GLMakie/rXGL8/src/screen.jl:247 [inlined]
  [7] global_gl_screen(resolution::Tuple{Int64, Int64}, visibility::Bool, tries::Int64)
    @ GLMakie ~/.julia/packages/GLMakie/rXGL8/src/screen.jl:394
  [8] global_gl_screen
    @ ~/.julia/packages/GLMakie/rXGL8/src/screen.jl:393 [inlined]
  [9] backend_display
    @ ~/.julia/packages/GLMakie/rXGL8/src/display.jl:2 [inlined]
 [10] display(scene::Scene; update::Bool)
    @ Makie ~/.julia/packages/Makie/xbI6d/src/display.jl:60
 [11] display
    @ ~/.julia/packages/Makie/xbI6d/src/display.jl:56 [inlined]
 [12] #display#905
    @ ~/.julia/packages/Makie/xbI6d/src/display.jl:52 [inlined]
 [13] display
    @ ~/.julia/packages/Makie/xbI6d/src/display.jl:52 [inlined]
 [14] #display#904
    @ ~/.julia/packages/Makie/xbI6d/src/display.jl:51 [inlined]
 [15] display(fap::Makie.FigureAxisPlot)
    @ Makie ~/.julia/packages/Makie/xbI6d/src/display.jl:51

It seems that the issue that was resolved is that drivers were expecting GLIBCXX_3.4.26 so things got out of sync again.

@j-fu workaround worked though: https://github.com/JuliaGL/GLFW.jl/issues/198#issuecomment-740124490

[roflmaostc]

@blegat I can confirm your issue and indeed the workaround works: #198 (comment).

My setup is Arch Linux, with integrated Intel GPU and

$ glxinfo | grep "OpenGL version"        
OpenGL version string: 4.6 (Compatibility Profile) Mesa 21.2.1

[DimitarVanguelov]

Hi, I'm on a laptop with an integrated AMD GPU and got this error, first time trying GLMakie: ERROR: GLFWError (VERSION_UNAVAILABLE): GLX: Failed to create context: GLXBadFBConfig. I tried the advice from comment and it worked (thanks @j-fu!).

My question is: is there a way to make this work in/with VS Code? Should something go in my startup.jl file or .zshrc?

Thanks in advance.