rust-dlopen icon indicating copy to clipboard operation
rust-dlopen copied to clipboard

Panic due to double SymInitializeW

Open branpk opened this issue 4 years ago • 1 comments

Rust on Windows has to call SymInitializeW before generating a backtrace, and it doesn't call SymCleanup afterward. If AddressInfoObtainer is then used, it calls SymInitializeW again, which fails because it has already been called.

Example code (set RUSTBACKTRACE=1):

use std::panic::catch_unwind;
use dlopen::raw::{AddressInfoObtainer, Library};

fn main() {
    let library = Library::open("example.dll").unwrap();
    let pointer: *const () = unsafe { library.symbol("foo") }.unwrap();

    catch_unwind(|| panic!()); // this generates a backtrace. Backtrace::capture() probably works too

    // Panics because SymInitializeW returns an error
    AddressInfoObtainer::new().obtain(pointer).unwrap();
}

There is some background on how the backtrace crate handled a similar conflict here (also explains why Rust doesn't call SymCleanup).

I think dlopen should:

  • Call SymInitializeW but ignore its return value
  • Never call SymCleanup

This behavior is awkward, but I think it's the only way to be compatible with how Rust uses SymInitializeW.

I can submit a PR for this if it's welcome.

branpk avatar Jun 08 '21 05:06 branpk

I implemented this in my fork dlopen2

OpenByteDev avatar Jul 07 '22 22:07 OpenByteDev