Linker error when used alongside SurrealDB with rocksdb backend

[alyti]

trafficstars

Hello, I am using this crate as part of deno_core, alongside surrealdb. This project and surrealdb's rocksdb are the cpp based dependencies here. When I try to run a very basic setup with both, I get a linker error:

  = note: liblibrocksdb_sys-4e44dbb3539271d1.rlib(backup_engine.o) : error LNK2005: "public: __cdecl std::exception_ptr::exception_ptr(class std::exception_ptr const &)" (??0exception_ptr@std@@QEAA@AEBV01@@Z) already defined in libv8-aa70c24694af9d2f.rlib(exception.obj)
          liblibrocksdb_sys-4e44dbb3539271d1.rlib(backup_engine.o) : error LNK2005: "public: __cdecl std::exception_ptr::exception_ptr(void)" (??0exception_ptr@std@@QEAA@XZ) already defined in libv8-aa70c24694af9d2f.rlib(exception.obj)
          liblibrocksdb_sys-4e44dbb3539271d1.rlib(backup_engine.o) : error LNK2005: "public: __cdecl std::exception_ptr::~exception_ptr(void)" (??1exception_ptr@std@@QEAA@XZ) already defined in libv8-aa70c24694af9d2f.rlib(exception.obj)
          liblibrocksdb_sys-4e44dbb3539271d1.rlib(backup_engine.o) : error LNK2005: "public: class std::exception_ptr & __cdecl std::exception_ptr::operator=(class std::exception_ptr const &)" (??4exception_ptr@std@@QEAAAEAV01@AEBV01@@Z) already defined in libv8-aa70c24694af9d2f.rlib(exception.obj)
          liblibrocksdb_sys-4e44dbb3539271d1.rlib(backup_engine.o) : error LNK2005: "public: __cdecl std::exception_ptr::operator bool(void)const " (??Bexception_ptr@std@@QEBA_NXZ) already defined in libv8-aa70c24694af9d2f.rlib(exception.obj)
             Creating library A:\Projects\crystaldb\target\release\deps\crystaldb.lib and object A:\Projects\crystaldb\target\release\deps\crystaldb.exp
          LINK : warning LNK4098: defaultlib 'libcmt.lib' conflicts with use of other libs; use /NODEFAULTLIB:library
          A:\Projects\crystaldb\target\release\deps\crystaldb.exe : fatal error LNK1169: one or more multiply defined symbols found

To reproduce, this is on following toolchain:

stable-x86_64-pc-windows-msvc (default)
rustc 1.70.0 (90c541806 2023-05-31)

With following cargo.toml:

[package]
name = "crystaldb"
version = "0.1.0"
edition = "2021"

[dependencies]
tokio = { version = "1.28", features = ["full"] }
surrealdb = { version = "1.0.0-beta.9", default-features = false, features = ["kv-rocksdb"] }
deno_core = { version = "0.191.0", default-features = false }
futures = "0.3.28"

And main.rs:

use surrealdb::engine::any::connect;

use deno_core::anyhow::Error;
use deno_core::op;
use deno_core::Extension;
use deno_core::JsRuntime;
use deno_core::OpState;
use deno_core::RuntimeOptions;
use futures::channel::mpsc;
use futures::stream::StreamExt;
use std::task::Poll;

type Task = Box<dyn FnOnce()>;

#[tokio::main]
async fn main() {
    let db = connect("file://temp.db").await.unwrap();
    println!("db: {:?}", db);

    let my_ext = Extension::builder("my_ext")
        .ops(vec![op_schedule_task::decl()])
        .event_loop_middleware(|state_rc, cx| {
            let mut state = state_rc.borrow_mut();
            let recv = state.borrow_mut::<mpsc::UnboundedReceiver<Task>>();
            let mut ref_loop = false;
            while let Poll::Ready(Some(call)) = recv.poll_next_unpin(cx) {
                call();
                ref_loop = true; // `call` can callback into runtime and schedule new callbacks :-)
            }
            ref_loop
        })
        .state(move |state| {
            let (tx, rx) = mpsc::unbounded::<Task>();
            state.put(tx);
            state.put(rx);
        })
        .build();

    // Initialize a runtime instance
    let mut js_runtime = JsRuntime::new(RuntimeOptions {
        extensions: vec![my_ext],
        ..Default::default()
    });

    // Schedule 10 tasks.
    js_runtime
        .execute_script_static(
            "<usage>",
            r#"for (let i = 1; i <= 10; i++) Deno.core.ops.op_schedule_task(i);"#,
        )
        .unwrap();
    js_runtime.run_event_loop(false).await.unwrap();
}

#[op]
fn op_schedule_task(state: &mut OpState, i: u8) -> Result<(), Error> {
    let tx = state.borrow_mut::<mpsc::UnboundedSender<Task>>();
    tx.unbounded_send(Box::new(move || println!("Hello, world! x{i}")))
        .expect("unbounded_send failed");
    Ok(())
}

It's a pretty basic example from Deno + a connect call to surrealdb which in turn calls rocksdb. If either of those are commented out, their respective CPP libs never get linked and the application builds. But if left as show above it will fail.

I've already made a tracking issue on rocksdb side: https://github.com/rust-rocksdb/rust-rocksdb/issues/795

[bartlomieju]

Does it compile correctly if you don't specify default-features = false for deno_core?

[alyti]

Does it compile correctly if you don't specify default-features = false for deno_core?

No, that was just my last ditch attempt hoping the default feature was breaking it.

edit: I even tried again by removing it, it actually yielded absolutely no change, not even forcing a recompile of deno_core or it's deps. So I forced it to rebuild via cargo clean, still linker error.

[mmastrac]

Does it fail if you use LLD for linking? (ie: linker = "rust-lld.exe" like the below config)

https://github.com/bevyengine/bevy/blob/653c10371e546176059bb779844c0569c0190b6b/.cargo/config_fast_builds#L16

[alyti]

Does it fail if you use LLD for linking? (ie: linker = "rust-lld.exe" like the below config)

https://github.com/bevyengine/bevy/blob/653c10371e546176059bb779844c0569c0190b6b/.cargo/config_fast_builds#L16

I got the same error, but in a different candy wrapping:

  = note: rust-lld: error: duplicate symbol: public: __cdecl std::exception_ptr::exception_ptr(void)
          >>> defined at liblibrocksdb_sys-4e44dbb3539271d1.rlib(backup_engine.o)
          >>> defined at libv8-3f830b40b203b7f8.rlib(exception.obj)

          rust-lld: error: duplicate symbol: public: __cdecl std::exception_ptr::exception_ptr(class std::exception_ptr const &)
          >>> defined at liblibrocksdb_sys-4e44dbb3539271d1.rlib(backup_engine.o)
          >>> defined at libv8-3f830b40b203b7f8.rlib(exception.obj)

          rust-lld: error: duplicate symbol: public: class std::exception_ptr & __cdecl std::exception_ptr::operator=(class std::exception_ptr const &)
          >>> defined at liblibrocksdb_sys-4e44dbb3539271d1.rlib(backup_engine.o)
          >>> defined at libv8-3f830b40b203b7f8.rlib(exception.obj)

          rust-lld: error: duplicate symbol: public: __cdecl std::exception_ptr::~exception_ptr(void)
          >>> defined at liblibrocksdb_sys-4e44dbb3539271d1.rlib(backup_engine.o)
          >>> defined at libv8-3f830b40b203b7f8.rlib(exception.obj)

          rust-lld: error: duplicate symbol: public: bool __cdecl std::exception_ptr::operator bool(void) const
          >>> defined at liblibrocksdb_sys-4e44dbb3539271d1.rlib(backup_engine.o)
          >>> defined at libv8-3f830b40b203b7f8.rlib(exception.obj)

At least it's more readable now I guess...

[alyti]

In similar vein I tried this and it actually worked. I am not sure what issues this might cause in runtime though.

[target.x86_64-pc-windows-msvc]
rustflags = ["-C", "link-arg=/FORCE:MULTIPLE"]

It also only applies to the msvc linker, so I am not sure if this would build on other platforms (if they have this issue? Idk) For the immediate time I'll keep testing it out and keep this issue open in case there's a proper solution available.