rust-rdkafka
rust-rdkafka copied to clipboard
Failed to compile on windows with `ssl-vendored` & `cmake-build` combination
Hey, I can't seem to get this to link on windows with a vendored ssl.
I.e, using:
rdkafka = { version = "0.29.0", features = ["ssl-vendored", "cmake-build"] }
I get the following output:
librdkafka_sys-54d8d0d1481a5f9e.rlib(rdkafka_ssl.obj) : error LNK2019: unresolved external symbol ENGINE_load_ssl_client_cert referenced in function rd_kafka_ssl_set_certs
librdkafka_sys-54d8d0d1481a5f9e.rlib(rdkafka_ssl.obj) : error LNK2019: unresolved external symbol ENGINE_free referenced in function rd_kafka_ssl_ctx_init
librdkafka_sys-54d8d0d1481a5f9e.rlib(rdkafka_ssl.obj) : error LNK2019: unresolved external symbol ENGINE_by_id referenced in function rd_kafka_ssl_ctx_init
librdkafka_sys-54d8d0d1481a5f9e.rlib(rdkafka_ssl.obj) : error LNK2019: unresolved external symbol ENGINE_ctrl_cmd_string referenced in function rd_kafka_ssl_ctx_init
librdkafka_sys-54d8d0d1481a5f9e.rlib(rdkafka_ssl.obj) : error LNK2019: unresolved external symbol ENGINE_init referenced in function rd_kafka_ssl_ctx_init
I am also seeing a lot of these errors:
libopenssl_sys-ba70b1b6d496cd9b.rlib(bio_b64.obj) : warning LNK4099: PDB 'ossl_static.pdb' was not found with 'libopenssl_sys-ba70b1b6d496cd9b.rlib(bio_b64.obj)' or at '..\target\debug\deps\ossl_static.pdb'; linking object as if no debug info
But I can confirm that ossl_static.pdb
is in the debug\build\openssl-sys-63196b440406ab23\out\openssl-build\install\lib
directory alongside the two lib files:
libcrypto.lib libssl.lib ossl_static.pdb
The link.exe
statement does include that lib directory:
"/LIBPATH:<snipped>\\target\\debug\\build\\openssl-sys-63196b440406ab23\\out\\openssl-build\\install\\lib"
I am fighting with the same on Linux
Ok, I've investigated this more and it appears to be because openssl-src
does not include the engine module by default on windows (& musl) targets: https://github.com/alexcrichton/openssl-src-rs/blob/97babda37a440e13a1492e95af25c7fbd33c20fe/src/lib.rs#L193-L201
So when it compiles the static version of openssl, it does so without the appropriate exports, hence why it's complaining.
If you add the following to Cargo.toml
it seems to resolve it (note: it needs to be under build-dependencies
), but I think we should include this in the rdkafka library when vendoring and so I'll raise a PR:
[build-dependencies]
openssl-src = { version = "111", features = ["force-engine"] }
@peter-formlogic
Cargo.toml is like follows but I am still getting an issue
rdkafka = { version = "0.29.0", features = ["cmake-build", "ssl-vendored"], default-features = false }
[build-dependencies]
openssl-src = { version = "111.25.3+1.1.1t", features = [ "force-engine" ] }
Error:
engines/e_afalg.c:24:10: fatal error: linux/version.h: No such file or directory
24 | #include <linux/version.h>
| ^~~~~~~~~~~~~~~~~
compilation terminated.
make[1]: *** [Makefile:5752: engines/e_afalg.o] Error 1
make[1]: *** Waiting for unfinished jobs....
make: *** [Makefile:177: build_libs] Error 2
thread 'main' panicked at '
Sounds like there are no linux headers installed wherever you're trying to run this.
You could try adding linux-headers-generic
as a package on ubuntu and kernel-headers-musl
on arch.
Given Cargo.toml
:
[package]
name = "rdtest"
version = "0.1.0"
edition = "2021"
[dependencies]
rdkafka = { version = "0.29.0", features = ["cmake-build", "ssl-vendored"]}
[build-dependencies]
openssl-src = { version = "111", features = [ "force-engine" ] }
and main.rs
:
use rdkafka::Timestamp;
fn main() {
let timestamp = Timestamp::now();
println!("{timestamp:?}");
}
The following docker file compiles & runs just fine:
FROM clux/muslrust
RUN ln -s /bin/g++ /bin/musl-g++
ADD Cargo.toml .
ADD src src
RUN cargo build --release
CMD ["target/x86_64-unknown-linux-musl/release/rdtest"]
Morning, after installing kernel-headers-musl
on arch now I get the following error: note: rust-lld: error: unable to find library -lresolv
Removing "gssapi-vendored", "libz-static"], default-features = false
did the trick, thanks so much :)
However, in Ubuntu the package you mention doesn't solve the error
in openssl-sys, they add this: https://github.com/sfackler/rust-openssl/blob/745b200425390ab2b0b5aa641a89237b2ff7426e/openssl-sys/Cargo.toml#L28
openssl-src = { version = "300.1.2", optional = true, features = ["legacy"] }
if you add
openssl-src = { version = "300", features = ["force-engine"] }
these configurations may conflict
so I add this:
[build-dependencies]
openssl-src = { version = "300", features = ["legacy", "force-engine"] }
it works