gettext-rs icon indicating copy to clipboard operation
gettext-rs copied to clipboard
verified publisher icon Koka

Tests segfault with musl libc and Rust 1.62.0

Open Minoru opened this issue 3 years ago • 10 comments

This can be reproduced outside of tests too, just create a crate, add gettext-rs to its dependencies, modify src/main.rs like so:

use gettextrs::*;

fn main() {
    println!("Hello, world!");

    setlocale(LocaleCategory::LcAll, "en_US.UTF-8");
    bindtextdomain("hellorust", "/usr/local/share/locale").unwrap();
    textdomain("hellorust").unwrap();

    println!("{}", gettext("Hi"));
}

and run it like so:

$ export TARGET=x86_64-unknown-linux-musl
$ rustup target add $TARGET
$ cargo run --target x86_64-unknown-linux-musl

GDB backtrace:

#0  0x0000000000000000 in ?? ()
#1  0x0000000000409d7f in libintl_rwlock_wrlock_multithreaded (lock=0x6952e0 <_nl_state_lock>)
    at /tmp/t392ca-0/gettext/gettext-tools/../gettext-runtime/intl/lock.c:429
#2  0x0000000000404f12 in set_binding_values (domainname=0x1188540 "hellorust", dirnamep=0x7ffee10a79f0,
    wdirnamep=0x0, codesetp=0x0) at /tmp/t392ca-0/gettext/gettext-tools/../gettext-runtime/intl/bindtextdom.c:98
#3  0x000000000040545b in libintl_bindtextdomain (domainname=0x1188540 "hellorust",
    dirname=0x11885a0 "/usr/local/share/locale")
    at /tmp/t392ca-0/gettext/gettext-tools/../gettext-runtime/intl/bindtextdom.c:444
#4  0x0000000000404e92 in bindtextdomain (domainname=0x1188540 "hellorust",
    dirname=0x11885a0 "/usr/local/share/locale")
    at /tmp/t392ca-0/gettext/gettext-tools/../gettext-runtime/intl/intl-compat.c:122
#5  0x0000000000402998 in gettextrs::bindtextdomain::h2cc1488e6d6831e5 (domainname=..., dirname=...)
    at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/gettext-rs-0.7.0/src/lib.rs:404
#6  0x0000000000401e32 in testing_gettext::main::h5197093dec0ba370 () at src/main.rs:7
#7  0x00000000004020ab in core::ops::function::FnOnce::call_once::h7d20ae0d30dd681d ()
    at /rustc/a8314ef7d0ec7b75c336af2c9857bfaf43002bfc/library/core/src/ops/function.rs:248
#8  0x0000000000402ffe in std::sys_common::backtrace::__rust_begin_short_backtrace::h7c61455a56e08df1 (
    f=0x401db0 <testing_gettext::main::h5197093dec0ba370>)
    at /rustc/a8314ef7d0ec7b75c336af2c9857bfaf43002bfc/library/std/src/sys_common/backtrace.rs:122
#9  0x0000000000401ce1 in std::rt::lang_start::_$u7b$$u7b$closure$u7d$$u7d$::hff3660182f52fe58 ()
    at /rustc/a8314ef7d0ec7b75c336af2c9857bfaf43002bfc/library/std/src/rt.rs:145
#10 0x0000000000420aa1 in core::ops::function::impls::_$LT$impl$u20$core..ops..function..FnOnce$LT$A$GT$$u20$for$u20$$RF$F$GT$::call_once::hebd650acbac1d4a4 ()
    at /rustc/a8314ef7d0ec7b75c336af2c9857bfaf43002bfc/library/core/src/ops/function.rs:280
#11 std::panicking::try::do_call::hecbd3eb037d46a8f () at library/std/src/panicking.rs:492
#12 std::panicking::try::he4917a79ebd580dc () at library/std/src/panicking.rs:456
#13 std::panic::catch_unwind::h556073795a30efab () at library/std/src/panic.rs:137
#14 std::rt::lang_start_internal::_$u7b$$u7b$closure$u7d$$u7d$::h1f79c9445318151b () at library/std/src/rt.rs:128
#15 std::panicking::try::do_call::h9ec5f34c23d02b33 () at library/std/src/panicking.rs:492
#16 std::panicking::try::h77ceb3f19415d425 () at library/std/src/panicking.rs:456
#17 std::panic::catch_unwind::hbdf745a6c0c2efdb () at library/std/src/panic.rs:137
#18 std::rt::lang_start_internal::h7e07fb0742296f5d () at library/std/src/rt.rs:128
#19 0x0000000000401cb0 in std::rt::lang_start::hd368f17672268ecd (
    main=0x401db0 <testing_gettext::main::h5197093dec0ba370>, argc=1, argv=0x7ffee10a8028)
    at /rustc/a8314ef7d0ec7b75c336af2c9857bfaf43002bfc/library/std/src/rt.rs:144
#20 0x0000000000401f9c in main () at library/std/src/sync/once.rs:494

Relevant part of lock.c:

int
glthread_rwlock_wrlock_multithreaded (gl_rwlock_t *lock)
{
  int err;

  err = pthread_mutex_lock (&lock->lock);        // <---- this is line 429
  if (err != 0)
    return err;
  /* Test whether no readers or writers are currently running.  */
  while (!(lock->runcount == 0))
    {
      /* This thread has to wait for a while.  Enqueue it among the
         waiting_writers.  */
      lock->waiting_writers_count++;
      err = pthread_cond_wait (&lock->waiting_writers, &lock->lock);
      if (err != 0)
        {
          lock->waiting_writers_count--;
          pthread_mutex_unlock (&lock->lock);
          return err;
        }
      lock->waiting_writers_count--;
    }
  lock->runcount--; /* runcount becomes -1 */
  return pthread_mutex_unlock (&lock->lock);
}

There is no reference to ptherad_mutex_lock in the binary:

$ nm target/x86_64-unknown-linux-musl/debug/test-gettext | grep pthread_mutex
0000000000057000 T pthread_mutexattr_gettype

With Rust 1.61.0 it works fine:

$ rustup override set 1.61.0
$ rustup target add $TARGET
$ cargo run --target $TARGET
   Compiling memchr v2.5.0
   Compiling temp-dir v0.1.11
   Compiling cc v1.0.73
   Compiling regex-syntax v0.6.26
   Compiling lazy_static v1.4.0
   Compiling aho-corasick v0.7.18
   Compiling gettext-sys v0.21.3
   Compiling regex v1.5.6
   Compiling locale_config v0.3.0
   Compiling gettext-rs v0.7.0
   Compiling test-gettext v0.1.0 (/dev/shm/test-gettext)
    Finished dev [unoptimized + debuginfo] target(s) in 2m 16s
     Running `target/x86_64-unknown-linux-musl/debug/test-gettext`
Hello, world!
Hi
$ nm target/x86_64-unknown-linux-musl/debug/test-gettext | grep pthread_mutex
0000000000058942 T pthread_mutexattr_destroy
0000000000057c40 T pthread_mutexattr_gettype
0000000000058945 T pthread_mutexattr_init
000000000005894e T pthread_mutexattr_settype
00000000000583e5 T pthread_mutex_destroy
00000000000583e8 T pthread_mutex_init
0000000000058400 T __pthread_mutex_lock
0000000000058400 W pthread_mutex_lock
000000000005841d t __pthread_mutex_timedlock
000000000005841d W pthread_mutex_timedlock
00000000000587a6 t __pthread_mutex_trylock
00000000000587a6 W pthread_mutex_trylock
000000000005860e t __pthread_mutex_trylock_owner
00000000000587c0 t __pthread_mutex_unlock
00000000000587c0 W pthread_mutex_unlock
0000000000017130 t _ZN4core3ptr75drop_in_place$LT$std..sys..unix..locks..pthread_mutex..PthreadMutexAttr$GT$17h465c0dcd952b22a9E

Rust's changelog mentions a few things that seem relevant, like:

  • https://github.com/rust-lang/rust/pull/96436/
  • https://github.com/rust-lang/rust/pull/95035/
  • https://github.com/rust-lang/rust/pull/95801/

It sounds like pthreads might have been linked by Rust for its own purposes, but now it's not, and that causes grief for us.

Next step here is to run cargo bisect-rustc to figure out when exactly the breakage was introduced.

Minoru avatar Jul 01 '22 21:07 Minoru

cargo bisect-rustc doesn't work for this:

error: toolchain 'bisector-ci-ec667fbcfcd29e2b5080f6be04e229356b27e600-x86_64-unknown-linux-gnu' does not support components: bisector-ci-ec667fbcfcd29e2b5080f6be04e229356b27e600-x86_64-unknown-linux-gnu is a custom toolchain

I'm now bisecting manually using nightlies.

Minoru avatar Jul 06 '22 19:07 Minoru

nightly-2022-04-04 is good, nightly-2022-04-05 is bad. I think this means that the offending commit was merged on 2022-04-04. There are only 23 PRs merged on that day. This one looks the most relevant, but it's not obvious if it's really the culprit, and what can we do to fix our code: https://github.com/rust-lang/rust/pull/95035

Minoru avatar Jul 06 '22 20:07 Minoru

These look much more relevant, but they don't quite fit the date:

  • https://github.com/rust-lang/rust/pull/95606
  • https://github.com/rust-lang/rust/pull/93901
  • https://github.com/rust-lang/rust/issues/95561

Minoru avatar Jul 06 '22 20:07 Minoru

I created a new reproducer which doesn't involve gettext-rs:

$ cargo new gettext-reproducer
$ cd gettext-reproducer
$ cargo add gettext-sys
$ cat src/main.rs
use std::ffi::CString;

// #[link(name = "pthread", kind = "static", modifiers = "+whole-archive")]
// extern {}

fn main() {
    unsafe {
        let domainname = CString::new("domainname").unwrap();
        let dirname = CString::new("/usr/share/locale").unwrap();
        gettext_sys::bindtextdomain(domainname.as_ptr(), dirname.as_ptr());
    }
    println!("Hello, world!");
}

It I uncomment the two commented lines, it fails to link:

$ cargo run --target x86_64-unknown-linux-musl
   Compiling gettext-reproducer v0.1.0 (/dev/shm/gettext-reproducer)
error: linking with `cc` failed: exit status: 1
  |
  = note: "cc" "-m64" "/home/minoru/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained/rcrt1.o" "/home/minoru/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained/crti.o" "/home/minoru/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained/crtbeginS.o" "/tmp/rustcy3gw0E/symbols.o" "/dev/shm/gettext-reproducer/target/x86_64-unknown-linux-musl/debug/deps/gettext_reproducer-c8d17bfb1412b708.144y2hu772v0h9sq.rcgu.o" "/dev/shm/gettext-reproducer/target/x86_64-unknown-linux-musl/debug/deps/gettext_reproducer-c8d17bfb1412b708.1d5w1ylhjo3js4xj.rcgu.o" "/dev/shm/gettext-reproducer/target/x86_64-unknown-linux-musl/debug/deps/gettext_reproducer-c8d17bfb1412b708.1dhm50a7ec7sns2m.rcgu.o" "/dev/shm/gettext-reproducer/target/x86_64-unknown-linux-musl/debug/deps/gettext_reproducer-c8d17bfb1412b708.1n71yfapacml5kzr.rcgu.o" "/dev/shm/gettext-reproducer/target/x86_64-unknown-linux-musl/debug/deps/gettext_reproducer-c8d17bfb1412b708.1xd6a4t6o1kmp1xt.rcgu.o" "/dev/shm/gettext-reproducer/target/x86_64-unknown-linux-musl/debug/deps/gettext_reproducer-c8d17bfb1412b708.25ztvzd7gyu667rf.rcgu.o" "/dev/shm/gettext-reproducer/target/x86_64-unknown-linux-musl/debug/deps/gettext_reproducer-c8d17bfb1412b708.28ew0kjmkdnudopd.rcgu.o" "/dev/shm/gettext-reproducer/target/x86_64-unknown-linux-musl/debug/deps/gettext_reproducer-c8d17bfb1412b708.2acgyux4jals14xt.rcgu.o" "/dev/shm/gettext-reproducer/target/x86_64-unknown-linux-musl/debug/deps/gettext_reproducer-c8d17bfb1412b708.2er7nw3b4khlkstf.rcgu.o" "/dev/shm/gettext-reproducer/target/x86_64-unknown-linux-musl/debug/deps/gettext_reproducer-c8d17bfb1412b708.2gdn5c8j64db1mo9.rcgu.o" "/dev/shm/gettext-reproducer/target/x86_64-unknown-linux-musl/debug/deps/gettext_reproducer-c8d17bfb1412b708.2gylji0vkwlaqjv3.rcgu.o" "/dev/shm/gettext-reproducer/target/x86_64-unknown-linux-musl/debug/deps/gettext_reproducer-c8d17bfb1412b708.2lpy83pdm8vwh61d.rcgu.o" "/dev/shm/gettext-reproducer/target/x86_64-unknown-linux-musl/debug/deps/gettext_reproducer-c8d17bfb1412b708.2m4q6rjs9gphh8aj.rcgu.o" "/dev/shm/gettext-reproducer/target/x86_64-unknown-linux-musl/debug/deps/gettext_reproducer-c8d17bfb1412b708.2zwu1u527knetnlk.rcgu.o" "/dev/shm/gettext-reproducer/target/x86_64-unknown-linux-musl/debug/deps/gettext_reproducer-c8d17bfb1412b708.3ikawvwavlmaj8r6.rcgu.o" "/dev/shm/gettext-reproducer/target/x86_64-unknown-linux-musl/debug/deps/gettext_reproducer-c8d17bfb1412b708.3lscxhgohlixuaic.rcgu.o" "/dev/shm/gettext-reproducer/target/x86_64-unknown-linux-musl/debug/deps/gettext_reproducer-c8d17bfb1412b708.3p17ms2bz9nh3r3a.rcgu.o" "/dev/shm/gettext-reproducer/target/x86_64-unknown-linux-musl/debug/deps/gettext_reproducer-c8d17bfb1412b708.4h5pmhs2cwc5z4gi.rcgu.o" "/dev/shm/gettext-reproducer/target/x86_64-unknown-linux-musl/debug/deps/gettext_reproducer-c8d17bfb1412b708.4q3wo9dmie0wpqh2.rcgu.o" "/dev/shm/gettext-reproducer/target/x86_64-unknown-linux-musl/debug/deps/gettext_reproducer-c8d17bfb1412b708.4qptb6zxaoyps0ju.rcgu.o" "/dev/shm/gettext-reproducer/target/x86_64-unknown-linux-musl/debug/deps/gettext_reproducer-c8d17bfb1412b708.4yrm68w1szlv9c6x.rcgu.o" "/dev/shm/gettext-reproducer/target/x86_64-unknown-linux-musl/debug/deps/gettext_reproducer-c8d17bfb1412b708.50avw5cjgavq2u8b.rcgu.o" "/dev/shm/gettext-reproducer/target/x86_64-unknown-linux-musl/debug/deps/gettext_reproducer-c8d17bfb1412b708.52hgtxsl9o0r2i5h.rcgu.o" "/dev/shm/gettext-reproducer/target/x86_64-unknown-linux-musl/debug/deps/gettext_reproducer-c8d17bfb1412b708.52smxi33x4fi30o9.rcgu.o" "/dev/shm/gettext-reproducer/target/x86_64-unknown-linux-musl/debug/deps/gettext_reproducer-c8d17bfb1412b708.546l64jpbyhc06no.rcgu.o" "/dev/shm/gettext-reproducer/target/x86_64-unknown-linux-musl/debug/deps/gettext_reproducer-c8d17bfb1412b708.5eproyci5t3qpiei.rcgu.o" "/dev/shm/gettext-reproducer/target/x86_64-unknown-linux-musl/debug/deps/gettext_reproducer-c8d17bfb1412b708.6zdjubscutzt8li.rcgu.o" "/dev/shm/gettext-reproducer/target/x86_64-unknown-linux-musl/debug/deps/gettext_reproducer-c8d17bfb1412b708.7u7b64242fhtxy4.rcgu.o" "/dev/shm/gettext-reproducer/target/x86_64-unknown-linux-musl/debug/deps/gettext_reproducer-c8d17bfb1412b708.9cc42fo599ip18z.rcgu.o" "/dev/shm/gettext-reproducer/target/x86_64-unknown-linux-musl/debug/deps/gettext_reproducer-c8d17bfb1412b708.wbae1hbet6jgo01.rcgu.o" "/dev/shm/gettext-reproducer/target/x86_64-unknown-linux-musl/debug/deps/gettext_reproducer-c8d17bfb1412b708.xhyyhgy2x5l2qy9.rcgu.o" "-Wl,--as-needed" "-L" "/dev/shm/gettext-reproducer/target/x86_64-unknown-linux-musl/debug/deps" "-L" "/dev/shm/gettext-reproducer/target/debug/deps" "-L" "/dev/shm/gettext-reproducer/target/x86_64-unknown-linux-musl/debug/build/gettext-sys-5380771ff93d9ad2/out/lib" "-L" "/home/minoru/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib" "-Wl,-Bstatic" "-Wl,--whole-archive" "-lpthread" "-Wl,--no-whole-archive" "/dev/shm/gettext-reproducer/target/x86_64-unknown-linux-musl/debug/deps/libgettext_sys-2ea14362b7b4edb0.rlib" "-Wl,--start-group" "/home/minoru/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/libstd-2c0ff204c579e132.rlib" "/home/minoru/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/libpanic_unwind-d0915314d07d6d36.rlib" "/home/minoru/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/libobject-99eb98cb8036c0ac.rlib" "/home/minoru/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/libmemchr-33a08e03781a81ec.rlib" "/home/minoru/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/libaddr2line-cf39d7df9a8f82a7.rlib" "/home/minoru/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/libgimli-28357830b23e2fef.rlib" "/home/minoru/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/librustc_demangle-0b91567160d0bd5f.rlib" "/home/minoru/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/libstd_detect-e4f469286cdd380f.rlib" "/home/minoru/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/libhashbrown-490774d955e3e803.rlib" "/home/minoru/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/libminiz_oxide-b93dd1f288009466.rlib" "/home/minoru/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/libadler-f9f641e713eb7197.rlib" "/home/minoru/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/librustc_std_workspace_alloc-f2a6926cd25d276e.rlib" "/home/minoru/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/libunwind-d667f5a0aef5d8cf.rlib" "-lunwind" "/home/minoru/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/libcfg_if-1ff32a3f72a57c9d.rlib" "/home/minoru/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/liblibc-43c83f88d784cf11.rlib" "-lc" "/home/minoru/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/liballoc-88a075cf6c777e0d.rlib" "/home/minoru/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/librustc_std_workspace_core-ab55f55aa6424772.rlib" "/home/minoru/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/libcore-a210d209c558541d.rlib" "-Wl,--end-group" "/home/minoru/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/libcompiler_builtins-c9e9dccd3a48e06a.rlib" "-Wl,-Bdynamic" "-Wl,--eh-frame-hdr" "-Wl,-znoexecstack" "-nostartfiles" "-L" "/home/minoru/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib" "-L" "/home/minoru/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained" "-o" "/dev/shm/gettext-reproducer/target/x86_64-unknown-linux-musl/debug/deps/gettext_reproducer-c8d17bfb1412b708" "-Wl,--gc-sections" "-static-pie" "-Wl,-zrelro,-znow" "-nodefaultlibs" "/home/minoru/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained/crtendS.o" "/home/minoru/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained/crtn.o"
  = note: /usr/bin/ld: /home/minoru/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained/libc.a(__errno_location.lo): in function `__pthread_self':
          /build/musl-cross-make/build/local/x86_64-linux-musl/obj_musl/../src_musl/arch/x86_64/pthread_arch.h:4: multiple definition of `__errno_location'; /usr/lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu/libpthread.a(errno-loc.o):./nptl/../csu/errno-loc.c:26: first defined here
          /usr/bin/ld: /home/minoru/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained/libc.a(pthread_create.lo): in function `__pthread_exit':
          /build/musl-cross-make/build/local/x86_64-linux-musl/obj_musl/../src_musl/src/thread/pthread_create.c:55: multiple definition of `__pthread_exit'; /usr/lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu/libpthread.a(pthread_exit.o):./nptl/pthread_exit.c:25: first defined here
          /usr/bin/ld: /home/minoru/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained/libc.a(sigaction.lo): in function `__libc_sigaction':
          /build/musl-cross-make/build/local/x86_64-linux-musl/obj_musl/../src_musl/src/signal/sigaction.c:27: multiple definition of `__libc_sigaction'; /usr/lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu/libpthread.a(sigaction.o):./nptl/../sysdeps/unix/sysv/linux/sigaction.c:43: first defined here
          /usr/bin/ld: /home/minoru/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained/libc.a(sigaction.lo): in function `__sigaction':
          /build/musl-cross-make/build/local/x86_64-linux-musl/obj_musl/../src_musl/src/signal/sigaction.c:81: multiple definition of `__sigaction'; /usr/lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu/libpthread.a(sigaction.o):./nptl/../nptl/sigaction.c:24: first defined here
          /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu/libpthread.a(elision-lock.o): warning: relocation against `_dl_x86_cpu_features' in read-only section `.text'
          /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu/libpthread.a(pthread_mutex_lock.o): in function `futex_fatal_error':
          ./nptl/../sysdeps/nptl/futex-internal.h:85: undefined reference to `__libc_fatal'
          /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu/libpthread.a(pthread_mutex_unlock.o): in function `futex_fatal_error':
          ./nptl/../sysdeps/nptl/futex-internal.h:85: undefined reference to `__libc_fatal'
          /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu/libpthread.a(pthread_mutex_cond_lock.o): in function `futex_fatal_error':
          ./nptl/../sysdeps/nptl/futex-internal.h:85: undefined reference to `__libc_fatal'
          /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu/libpthread.a(pthread_rwlock_rdlock.o): in function `futex_fatal_error':
          ./nptl/../sysdeps/nptl/futex-internal.h:85: undefined reference to `__libc_fatal'
          /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu/libpthread.a(pthread_rwlock_wrlock.o): in function `futex_fatal_error':
          ./nptl/../sysdeps/nptl/futex-internal.h:85: undefined reference to `__libc_fatal'
          /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu/libpthread.a(pthread_rwlock_unlock.o):./nptl/../sysdeps/nptl/futex-internal.h:85: more undefined references to `__libc_fatal' follow
          /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu/libpthread.a(unwind.o): in function `unwind_stop':
          ./nptl/unwind.c:94: undefined reference to `__libc_longjmp'
          /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu/libpthread.a(unwind.o): in function `unwind_cleanup':
          ./nptl/unwind.c:105: undefined reference to `__libc_fatal'
          /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu/libpthread.a(write.o): in function `__libc_write':
          ./nptl/../sysdeps/unix/sysv/linux/write.c:26: undefined reference to `errno'
          /usr/bin/ld: ./nptl/../sysdeps/unix/sysv/linux/write.c:26: undefined reference to `errno'
          /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu/libpthread.a(close.o): in function `__close':
          ./nptl/../sysdeps/unix/sysv/linux/close.c:27: undefined reference to `errno'
          /usr/bin/ld: ./nptl/../sysdeps/unix/sysv/linux/close.c:27: undefined reference to `errno'
          /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu/libpthread.a(lseek64.o): in function `__lseek64':
          ./nptl/../sysdeps/unix/sysv/linux/lseek64.c:36: undefined reference to `errno'
          /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu/libpthread.a(open64.o):./nptl/../sysdeps/unix/sysv/linux/open64.c:48: more undefined references to `errno' follow
          /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu/libpthread.a(tpp.o): in function `__init_sched_fifo_prio':
          ./nptl/tpp.c:45: undefined reference to `__sched_get_priority_max'
          /usr/bin/ld: ./nptl/tpp.c:47: undefined reference to `__sched_get_priority_min'
          /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu/libpthread.a(tpp.o): in function `__pthread_tpp_change_priority':
          ./nptl/tpp.c:149: undefined reference to `__sched_setscheduler'
          /usr/bin/ld: ./nptl/tpp.c:150: undefined reference to `errno'
          /usr/bin/ld: ./nptl/tpp.c:126: undefined reference to `__sched_getparam'
          /usr/bin/ld: ./nptl/tpp.c:127: undefined reference to `errno'
          /usr/bin/ld: ./nptl/tpp.c:134: undefined reference to `__sched_getscheduler'
          /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu/libpthread.a(tpp.o): in function `__init_sched_fifo_prio':
          ./nptl/tpp.c:45: undefined reference to `__sched_get_priority_max'
          /usr/bin/ld: ./nptl/tpp.c:47: undefined reference to `__sched_get_priority_min'
          /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu/libpthread.a(tpp.o): in function `__pthread_tpp_change_priority':
          ./nptl/tpp.c:136: undefined reference to `errno'
          /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu/libpthread.a(tpp.o): in function `__pthread_current_priority':
          ./nptl/tpp.c:174: undefined reference to `__sched_getparam'
          /usr/bin/ld: ./nptl/tpp.c:182: undefined reference to `__sched_getscheduler'
          /usr/bin/ld: ./nptl/tpp.c:182: undefined reference to `__sched_getscheduler'
          /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu/libpthread.a(elision-lock.o): in function `elision_init':
          ./nptl/../sysdeps/unix/sysv/linux/x86/elision-conf.c:110: undefined reference to `__tunable_get_val'
          /usr/bin/ld: ./nptl/../sysdeps/unix/sysv/linux/x86/elision-conf.c:112: undefined reference to `__tunable_get_val'
          /usr/bin/ld: ./nptl/../sysdeps/unix/sysv/linux/x86/elision-conf.c:114: undefined reference to `__tunable_get_val'
          /usr/bin/ld: ./nptl/../sysdeps/unix/sysv/linux/x86/elision-conf.c:116: undefined reference to `__tunable_get_val'
          /usr/bin/ld: ./nptl/../sysdeps/unix/sysv/linux/x86/elision-conf.c:118: undefined reference to `__tunable_get_val'
          /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu/libpthread.a(elision-lock.o): in function `do_set_elision_enable':
          ./nptl/../sysdeps/unix/sysv/linux/x86/elision-conf.c:66: undefined reference to `_dl_x86_cpu_features'
          /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu/libpthread.a(errno-loc.o): in function `__errno_location':
          ./nptl/../csu/errno-loc.c:26: undefined reference to `errno'
          /usr/bin/ld: read-only segment has dynamic relocations
          collect2: error: ld returned 1 exit status

  = help: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified
  = note: use the `-l` flag to specify native libraries to link
  = note: use the `cargo:rustc-link-lib` directive to specify the native libraries to link with Cargo (see https://doc.rust-lang.org/cargo/reference/build-scripts.html#cargorustc-link-libkindname)

error: could not compile `gettext-reproducer` due to previous error

The intent of the commented lines is to force linking the pthread library. Apparently that's not the way forward =\

Minoru avatar Jul 07 '22 19:07 Minoru

I tried to approach this from an opposite direction: write a tiny C library, wrap it in a Rust crate, and use it from a different Rust crate. The result is in https://github.com/Minoru/gettextrs-musl-segfault-mre, and it does not exhibit the problem. It runs just fine. Puzzling!

Minoru avatar Jul 10 '22 17:07 Minoru

I give up. As documented above, I failed to create a minimal reproducible example, so I won't be reporting this upstream; that would have been akin to asking them to fix our crate for us.

Some ideas that I didn't try:

  • pass --print=native-static-libs flag to rustc to check if different compiler versions pass different libraries to the linker
  • ~~write a small binary crate that links directly to gettext-sys. That would have excluded gettext-rs crate from consideration, hopefully making this easier to debug~~ actually, I did, and it doesn't exhibit the probelm
  • look at how cc crate links with native dependencies. Compare to our build.rs to see if we're doing anything wrong

Minoru avatar Jul 19 '22 19:07 Minoru

A couple suggestions from Mastodon:

  • try 1.62.1 that was released a few days ago -- I did, it still segfaults
  • check the order in which libraries are linked -- I didn't try that yet

Minoru avatar Jul 21 '22 19:07 Minoru 

#[link(name = "pthread", kind = "static", modifiers = "+whole-archive")]
extern {}

This doesn't work because musl's pthread implementation sits in libc, not in libpthread, which is empty (see the first question here). Your example works (stops segfaulting) if you do name = "c" to include libc with +whole-archive, instead of libpthread.

Including libc with +whole-archive is probably not a good solution, but that this makes the example work probably helps with figuring out what the problem is.

It reminds me a bit of the problem that's described in this comment: https://github.com/rust-lang/rust/issues/94564#issuecomment-1065185676

I hope that's useful.

m-ou-se avatar Sep 19 '22 13:09 m-ou-se

Thanks! This indeed helps with the example, and works even without +whole-archive. It also helps with tests: if I stick that extern somewhere in the tests sub-module in gettext-rs, the tests stop segfaulting.

However, it doesn't help with my initial reproducer from the top post: adding an extern leads to linker errors because of multiple definitions of gettext, dgettext and other gettext functions. This does make sense since musl contains its own gettext implementation. The only way to get rid of that error is to enable gettext-system feature, but in that case, the extern becomes superfluous.

Same goes for adding the extern to the library itself: I get "error: could not find native static library c, perhaps an -L flag is missing?" and liking errors due to multiple definitions. It's weird that it can't find a library that is obviously already linked somehow =\

Now I'm scratching my head over how it could work in the first place. These linking errors should have popped up earlier too, regardless of Rust version. Baffling!

Minoru avatar Oct 08 '22 16:10 Minoru

i can't really reproduce this on a musl-hosted system (alpine) that by default compiles without +crt-static (dynamic) and has some weird extra rust patches. same goes for adding that to make it static. the only way to get duplicate symbol errors is to both add +crt-static and any kind of #[link(name = "c".. at the same time, and nothing ever segfaults (it's possible to get an abort by trying to extern kind = "static" at the same time as -crt-static, but that's it)

overall, my guess is something to do with the link order or the way the default rust triple (the one you get from rustup) links things, as by default the static binaries it makes don't really work on a musl system (it defaults to +crt-static and the binaries don't work for some things, iirc the easiest reproduction is trying to use the gtk4 crate). sadly, the way i fixed static-linking in the alpine patches in the first place was throwing random link args all over the rust build system as i have no idea how they really translate to link argument order to cc in the end.

tl;dr this is probably a default-musl-triple issue but it's hard to get a solid reproduction as to why it happens

nekopsykose avatar Nov 10 '22 14:11 nekopsykose