Yggdrasil Dyninst: New package

Oct 03 '23 00:10 eschnett

While the build works fine, the checks afterwards crash with

[ Info: Checking shared library lib/libcommon.so.12.3.0
  | free(): invalid pointer
  |  
signal (6): Aborted

(libcommon is one of the libraries that's built.) How would I debug this?

Oct 05 '23 19:10 eschnett

It's the dlopening that's failing, maybe the init function of the library does something nasty?

Oct 05 '23 21:10 giordano

@giordano I can successfully dlopen each of the shared libraries from within the build script. However, this still fails from within Julia. Do you have an idea why this might be?

I also looked at the libraries' initialization routines but that's a bit hopeless because these are large C++ libraries and I don't know which objects would be created or which functions would be called.

Oct 10 '23 22:10 eschnett

To leave some breadcrumbs, the init function of the shared library is marked with __attribute__((constructor)) and it's called runDYNINSTBaseInit: https://github.com/dyninst/dyninst/blob/ddd2315b562efdca3fe6aa25d6da2a3842c293fa/dyninstAPI_RT/src/RTlinux.c#L644-L661. But this function only calls the function r_debugCheck() and DYNINSTBaseInit: the former looks relatively innocuous (only running an assert: https://github.com/dyninst/dyninst/blob/ddd2315b562efdca3fe6aa25d6da2a3842c293fa/dyninstAPI_RT/src/RTlinux.c#L412), but the latter is more convoluted because it then calls other functions: https://github.com/dyninst/dyninst/blob/ddd2315b562efdca3fe6aa25d6da2a3842c293fa/dyninstAPI_RT/src/RTcommon.c#L163-L174

Oct 10 '23 23:10 giordano

This is what we do during audit to dlopen the library: https://github.com/JuliaPackaging/BinaryBuilder.jl/blob/54284653349b50eef5ad50b0c579659c38ef99e6/src/Auditor.jl#L150-L159. Not really much (using Libdl; dlopen(/path/to/library))

Oct 10 '23 23:10 giordano

I saw the call in Auditor.jl, and I am using the same flags when calling dlopen from C in the build script, which is working (no segfault). I think I don't understand the difference between the build environment and the environment in which the auditor is running in CI. I assume the build script is running in a Docker container that (somehow) makes things work? Maybe a system library is different?

Oct 11 '23 00:10 eschnett

There are different libstdc++ libraries:

libstdc++.so.6 => /workspace/destdir/lib/libstdc++.so.6 (0x00007fa5b1617000)
libstdc++.so.6 => /usr/lib/csl-glibc-x86_64/libstdc++.so.6 (0x00007f99a555a000)

The first is used by Dyninst, the second by Boost. That doesn't look healthy.

Oct 11 '23 17:10 eschnett

While the dlopen in the auditor fails, I can:

generate a package locally
using Dyninst_jll
dlopen(Dyninst_jll.libcommon) and this does not segfault. I think this is somehow a problem in the auditor.

Oct 19 '23 16:10 eschnett

@giordano I disabled the auditor. I think (see above) that the generated shared libraries are actually working fine.

Oct 19 '23 18:10 eschnett

I'm not incredibly happy about disabling the auditor, it does more stuff than just dlopening the library, like ensure the runpath is set correctly, the soname is set, etc...

Oct 20 '23 10:10 giordano

I'm still convinced they do something dodgy in their init function: https://github.com/JuliaPackaging/Yggdrasil/pull/7473#issuecomment-1756409554. If we have problems during audit here, we can have problems elsewhere, if dlopen is failing we should fix it, rather ignoring it.

Oct 20 '23 10:10 giordano

The current options actually only set dont_dlopen, they don't explicitly disable the auditor. I assume this means that the auditor is still running some other tests?

Apart from setting a few global variables Dyninst installs a signal handler in DYNINSTinitializeTrapHandler:

int DYNINSTinitializeTrapHandler(void)
{
   int result;
   struct sigaction new_handler;
   int signo = SIGTRAP;

   // If environment variable DYNINST_SIGNAL_TRAMPOLINE_SIGILL is set,
   // we use SIGILL as the signal for signal trampoline.
   // The mutatee has to be generated with DYNINST_SIGNAL_TRAMPOLINE_SIGILL set
   // so that the mutator will generates illegal instructions as trampolines.
   if (getenv("DYNINST_SIGNAL_TRAMPOLINE_SIGILL")) {
      signo = SIGILL;
   }

   new_handler.sa_sigaction = dyninstTrapHandler;
   //new_handler.sa_restorer = NULL; obsolete
   sigemptyset(&new_handler.sa_mask);
   new_handler.sa_flags = SA_SIGINFO | SA_NODEFER;
   
   result = sigaction(signo, &new_handler, NULL);
   return (result == 0) ? 1 /*Success*/ : 0 /*Fail*/ ;
}

Maybe Julia doesn't like SIGTRAP being caught?

Oct 21 '23 15:10 eschnett