Unable to run on Musl libc, "can't get libdl.so"

[PureTryOut]

I've been trying to get MangoHud to run on Musl libc systems to use with FOSS games compiled for this libc, but so far I've been unable to do so.

> mangohud --dlsym target/debug/leafish
can't get libdl.so

I'm not sure what this "libdl.so" is, but I'm assuming it's a glibc thing? MangoHud compiles fine with Musl however, so not sure what's up.

[flightlessmango]

Yes I believe it's part of libc. We (at least partly) use it for dlopen so we can load libraries at runtime

From the musl manual Additional files libm.a, librt.a, libpthread.a, libcrypt.a, libutil.a, libxnet.a, libresolv.a, and libdl.a are provided in $(libdir) as empty library archives. They contain no code, but are present to satisfy the POSIX requirement that options of the form -lm, -lpthread, etc. be accepted by the c99 compiler.

[PureTryOut]

Shit, so I guess that means it won't work. At least till they actually implement it rather than just being empty files...

[magiruuvelvet]

How about just replacing libdl with libc when calling dlopen() in the source code? The functions are there, just the filename is different. musl libc can't be detected using compiler macros though, so a build-time check would be needed if adjusting the source.

EDIT: As alternative a symlink to libc.so called libdl.so should work too.

[PureTryOut]

musl libc can't be detected using compiler macros though

Well, indirectly it can.

#if defined(__LINUX__) && !defined(__GLIBC_)

That basically comes down to Musl. Well and I suppose uclibc, but who uses that on a platform where they would want to use MangoHud? :thinking:

[XRevan86]

glibc 2.34+ also moved dlopen and dlsym to libc.so, so I'd imagine the solution would be something like this:

--- a/src/real_dlsym.cpp
+++ b/src/real_dlsym.cpp
@@ -19,10 +19,17 @@ void get_real_functions()
 {
     eh_obj_t libdl;
 
+#if !defined(__GLIBC_) || __GLIBC_PREREQ(2, 34)
+    if (eh_find_obj(&libdl, "*libc.so*")) {
+        fprintf(stderr, "can't get libc.so\n");
+        exit(1);
+    }
+#else
     if (eh_find_obj(&libdl, "*libdl.so*")) {
         fprintf(stderr, "can't get libdl.so\n");
         exit(1);
     }
+#endif
 
     if (eh_find_sym(&libdl, "dlopen", (void **) &__dlopen)) {
         fprintf(stderr, "can't get dlopen()\n");

[jackun]

I think it still needs libdl path for compatibility. Does https://github.com/flightlessmango/MangoHud/commit/ced84ec526d37097b66aa89e1d93e87d7e9a2780 help?

[PureTryOut]

It sadly does not.

> mangohud --dlsym target/release/leafish
MANGOHUD: Cannot find libdl.so and libc.so

However /lib/libc.musl-x86_64.so.1 exists, so not sure why it can't find it.

EDIT: Oh wait, it searches for *libc.so* while in this case it should be *libc*.so*

[jackun]

:weary: Ah yes, but probably libc.*.so* so it doesn't load libcrypto or something instead.

[PureTryOut]

I just modified your commit to do just that and I get the same result sadly.

However checking eh_find_obj() it has a comment:

/* This function uses glibc-specific dl_iterate_phdr().
        Another way could be parsing /proc/self/exe or using
        pmap() on Solaris or *BSD */

So I assume this will never work on Musl?

[jackun]

Try 2: https://github.com/flightlessmango/MangoHud/tree/dlsym_fs Helps any? Glibc 2.34 seemed to work at least.

e: dl_iterate_phdr probably still blocks?

[kerossin]

Try 2: https://github.com/flightlessmango/MangoHud/tree/dlsym_fs Helps any? Glibc 2.34 seemed to work at least.

e: dl_iterate_phdr probably still blocks?

Hi,

I'm on openSUSE Tumbleweed (2.34-1.2 currently) and I was getting can't get libdl.so when trying to run MangoHud with --dlsym. Using the libraries compiled from the dlsym_fs branch seem to have fixed it.

[PureTryOut]

@jackun I'd love to try but sadly that branch fails to compile for me (before I only compiled latest release, 2.6.5):

./src/overlay.cpp: In function 'void render_mpris_metadata(overlay_params&, mutexed_metadata&, uint64_t)':
../src/overlay.cpp:355:33: error: expected unqualified-id before '::' token
  355 |             catch (const fmt::v7::format_error& err)
      |                                 ^~
../src/overlay.cpp:355:33: error: expected ')' before '::' token
  355 |             catch (const fmt::v7::format_error& err)
      |                   ~             ^~
      |                                 )
../src/overlay.cpp:355:33: error: expected '{' before '::' token
  355 |             catch (const fmt::v7::format_error& err)
      |                                 ^~
../src/overlay.cpp:355:35: error: '::format_error' has not been declared; did you mean 'fmt::v8::format_error'?
  355 |             catch (const fmt::v7::format_error& err)
      |                                   ^~~~~~~~~~~~
      |                                   fmt::v8::format_error
In file included from /usr/include/spdlog/fmt/fmt.h:26,
                 from /usr/include/spdlog/common.h:36,
                 from /usr/include/spdlog/spdlog.h:12,
                 from ../src/overlay.cpp:6:
/usr/include/fmt/format.h:748:15: note: 'fmt::v8::format_error' declared here
  748 | class FMT_API format_error : public std::runtime_error {
      |               ^~~~~~~~~~~~
../src/overlay.cpp:355:49: error: 'err' was not declared in this scope; did you mean 'spdlog::level::err'?
  355 |             catch (const fmt::v7::format_error& err)
      |                                                 ^~~
      |                                                 spdlog::level::err
In file included from /usr/include/spdlog/spdlog.h:12,
                 from ../src/overlay.cpp:6:
/usr/include/spdlog/common.h:167:5: note: 'spdlog::level::err' declared here
  167 |     err = SPDLOG_LEVEL_ERROR,
      |     ^~~

[jackun]

For some reason it is using system fmt v8, spdlog 1.8.5 needs v7.

[PureTryOut]

This is spdlog 1.9.2 from the system, which pulls in it's own fmt. Is that too new for mangohud?

[jackun]

Ah, nevermind. It's my own function, try changing ~~v7 to v8~~ to const fmt::format_error& err at line 355 in src/overlay.cpp.

[PureTryOut]

That worked, getting close.

../src/overlay_params.cpp: In function 'void parse_overlay_config(overlay_params*, const char*)':
../src/overlay_params.cpp:776:72: error: 'getpid' was not declared in this scope
  776 |          HUDElements.gamemode_bol = dbusmgr::dbus_mgr.gamemode_enabled(getpid());
      |                                                                        ^~~~~~

[jackun]

Pull again but dl_iterate_phdr is borked, doesn't list libc.so though it is in /proc/../map_files, on Void at least :weary:

[PureTryOut]

⋊> ~/D/G/l/Leafish on main ◦ mangohud --dlsym target/release/leafish                                         10:55:15
MANGOHUD: Can't get dlopen() and dlsym()

[akien-mga]

See #693. I had the same issue with mangohud v0.6.5 with glibc, and updating to v0.6.6-1 solved it.

[kibasnowpaw]

See #693. I had the same issue with mangohud v0.6.5 with glibc, and updating to v0.6.6-1 solved it.

yeah v0.6.6-1 Fix it. https://github.com/flightlessmango/MangoHud/releases/tag/v0.6.6-1

[CordBlaster]

This problem exists in Ubuntu 22.04 (jammy jellyfish). This bug is probably related: https://bugs.launchpad.net/ubuntu/+source/mangohud/+bug/1993862

[maribu]

In https://gitlab.alpinelinux.org/alpine/aports/-/merge_requests/46458 I'm trying to package this for Alpine.

With the following patch applied, it almost works:

--- a/src/meson.build
+++ b/src/meson.build
@@ -3,10 +3,10 @@
 # Needs prefix for configure_file()
 if get_option('append_libdir_mangohud')
   libdir_mangohud = join_paths(get_option('prefix'), get_option('libdir'), 'mangohud')
-  ld_libdir_mangohud = get_option('prefix') + '/\$LIB/mangohud/'
+  ld_libdir_mangohud = join_paths(get_option('prefix'), get_option('libdir'), 'mangohud') + '/'
 else
   libdir_mangohud = join_paths(get_option('prefix'), get_option('libdir'))
-  ld_libdir_mangohud = get_option('prefix') + '/\$LIB/'
+  ld_libdir_mangohud = join_paths(get_option('prefix'), get_option('libdir')) + '/'
 endif
 
 conf_data = configuration_data()
--- a/src/real_dlsym.cpp
+++ b/src/real_dlsym.cpp
@@ -26,6 +26,7 @@
 #endif
         "*libc.so*",
         "*libc.*.so*",
+        "*ld-musl-*.so*",
     };
 
     for (size_t i = 0; i < sizeof(libs) / sizeof(*libs); i++)

However, this check fails (for both obj->strtab and obj->symtab):

	/* This is here to catch b0rken headers (vdso) */
	if ((eh_check_addr(obj, (const void *) obj->strtab)) |
	    (eh_check_addr(obj, (const void *) obj->symtab)))
		return ENOTSUP;

So, for OpenGL no luck yet, but with Vulkan it works fine with Alpine/musl.

Has anyone an idea why obj->strtab and obj->symtab are holding invalid addresses on Alpine/musl?

[apprehensions]

Has anyone an idea why obj->strtab and obj->symtab are holding invalid addresses on Alpine/musl?

I was looking to use MangoHud on Alpine, but unfortunately this is still a problem, even on 0.6.9.

Void Linux has MangoHud packaged for musl, maybe it works for them as they dont include such a patch for dlsym.

[Johnnynator]

Has anyone an idea why obj->strtab and obj->symtab are holding invalid addresses on Alpine/musl?

See elf(5)

       d_ptr  This member represents program virtual addresses.  When
              interpreting these addresses, the actual address should be
              computed based on the original file value and memory base
              address.  Files do not contain relocation entries to fixup
              these addresses.

The way I'm reading this, is that d_ptr is supposed to only hold an offset. Adding obj->addr to all of them does fix the code for musl. Glibc implements this different than documented in elf(5).

Fix for this is available in #1133