support for openssl 3.0.0

[satya-rajesh]

Platform: photon https://github.com/vmware/photon/

When compiled with openssl 3.0.0 alpha17, librdkafka does not build:

rdkafka_sasl_scram.c: In function 'rd_kafka_sasl_scram_conf_validate': rdkafka_sasl_scram.c:886:44: error: 'SHA1' undeclared (first use in this function) 886 | rk->rk_conf.sasl.scram_H = SHA1; | ^~~~ rdkafka_sasl_scram.c:886:44: note: each undeclared identifier is reported only once for each function it appears in rdkafka_sasl_scram.c:890:44: error: 'SHA256' undeclared (first use in this function); did you mean 'SHA256_CTX'? 890 | rk->rk_conf.sasl.scram_H = SHA256; | ^~~~~~ | SHA256_CTX rdkafka_sasl_scram.c:894:44: error: 'SHA512' undeclared (first use in this function) 894 | rk->rk_conf.sasl.scram_H = SHA512; | ^~~~~~ rdunittest.c: In function 'unittest_sysqueue': rdunittest.c:202:36: warning: 'tqh[0]' may be used uninitialized in this function [-Wmaybe-uninitialized] 202 | RD_UT_ASSERT(TAILQ_LAST(tqh[0], ut_tq_head)->v == args->exp[totcnt-1], | ^ make[1]: *** [../mklove/Makefile.base:95: rdkafka_sasl_scram.o] Error 1 make[1]: *** Waiting for unfinished jobs....

[satya-rajesh]

Any update on this ?

[edenhill]

Isn’t OpenSSL 3.0 still in beta?

fre 4 juni 2021 kl. 08:02 skrev satya-rajesh @.***>:

Any update on this ?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/edenhill/librdkafka/issues/3408#issuecomment-854384287, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAEAFPTPNCG63PV46NKMRMLTRBT6ZANCNFSM4524LNYQ .

[satya-rajesh]

Isn’t OpenSSL 3.0 still in beta? fre 4 juni 2021 kl. 08:02 skrev satya-rajesh @.***>:

Openssl 3.0 beta1 is released: https://github.com/openssl/openssl/releases/tag/openssl-3.0.0-beta1

[edenhill]

Okay, PRs welcome!

fre 4 juni 2021 kl. 12:23 skrev satya-rajesh @.***>:

Isn’t OpenSSL 3.0 still in beta? fre 4 juni 2021 kl. 08:02 skrev satya-rajesh @.***>:

As per openssl team, it is on June 30: https://github.com/openssl/openssl/milestone/17

Appreciate if librdkafka can start looking into this to support openssl 3.0.0 (latest is alpha17) so that by beta a patch can be provided to unblock issues.

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub https://github.com/edenhill/librdkafka/issues/3408#issuecomment-854588162, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAEAFPTDCDSSSR5W5N3LFDLTRCSTJANCNFSM4524LNYQ .

[viccpp]

OpenSSL 3.0 has been released

[CyberCMRX]

I don't get my librdkafka application compiled with OpenSSL 3.0: rdkafka.lib(rdkafka_ssl.obj) : error LNK2001: unresolved external symbol SSL_get_peer_certificate

[juchem]

if OpenSSL 3.0 is compiled with no-deprecated, which disables deprecated APIs, there are additional linker errors regarding the APIs that librdkafka should migrate away from:

Creating program kcat
gcc -D_GLIBCXX_USECXX11_ABI=0 -D_GLIBCXX_USECXX11_ABI=0 -g -O2 -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -Wcast-align  kcat.o format.o tools.o input.o -o kcat -L/usr/local/lib /usr/local/lib/librdkafka.a -lpthread -lrt -ldl -lm -lssl -lcrypto -lsqlite3 -lzstd -llz4 -lz -lresolv -lcrypt -ldl
/usr/bin/ld: /usr/local/lib/librdkafka.a(rdkafka_ssl.c.o): in function `rd_kafka_ssl_error.isra.0':
rdkafka_ssl.c:(.text+0x17f): undefined reference to `ERR_get_error_line_data'
/usr/bin/ld: /usr/local/lib/librdkafka.a(rdkafka_ssl.c.o): in function `rd_kafka_ssl_set_certs':
rdkafka_ssl.c:(.text+0x79e): undefined reference to `ENGINE_load_ssl_client_cert'
/usr/bin/ld: /usr/local/lib/librdkafka.a(rdkafka_ssl.c.o): in function `rd_kafka_ssl_last_error_str':
rdkafka_ssl.c:(.text+0x17f5): undefined reference to `ERR_peek_last_error_line_data'
/usr/bin/ld: rdkafka_ssl.c:(.text+0x1841): undefined reference to `ERR_func_error_string'
/usr/bin/ld: /usr/local/lib/librdkafka.a(rdkafka_ssl.c.o): in function `rd_kafka_transport_ssl_handshake':
rdkafka_ssl.c:(.text+0x1e87): undefined reference to `SSL_get_peer_certificate'
/usr/bin/ld: /usr/local/lib/librdkafka.a(rdkafka_ssl.c.o): in function `rd_kafka_ssl_ctx_init':
rdkafka_ssl.c:(.text+0x22e3): undefined reference to `ENGINE_free'
/usr/bin/ld: rdkafka_ssl.c:(.text+0x230f): undefined reference to `ENGINE_by_id'
/usr/bin/ld: rdkafka_ssl.c:(.text+0x2334): undefined reference to `ENGINE_ctrl_cmd_string'
/usr/bin/ld: rdkafka_ssl.c:(.text+0x2356): undefined reference to `ENGINE_ctrl_cmd_string'
/usr/bin/ld: rdkafka_ssl.c:(.text+0x2373): undefined reference to `ENGINE_ctrl_cmd_string'
/usr/bin/ld: rdkafka_ssl.c:(.text+0x2385): undefined reference to `ENGINE_init'
/usr/bin/ld: rdkafka_ssl.c:(.text+0x23c8): undefined reference to `ENGINE_by_id'
/usr/bin/ld: rdkafka_ssl.c:(.text+0x2534): undefined reference to `ENGINE_free'
/usr/bin/ld: rdkafka_ssl.c:(.text+0x255c): undefined reference to `ENGINE_free'
/usr/bin/ld: rdkafka_ssl.c:(.text+0x2601): undefined reference to `ENGINE_free'
/usr/bin/ld: rdkafka_ssl.c:(.text+0x2664): undefined reference to `ENGINE_free'
/usr/bin/ld: /usr/local/lib/librdkafka.a(rdkafka_ssl.c.o): in function `rd_kafka_ssl_ctx_term':
rdkafka_ssl.c:(.text+0x20fb): undefined reference to `ENGINE_free'
collect2: error: ld returned 1 exit status

FWIW, there's a migration guide for OpenSSL 3: https://github.com/openssl/openssl/blob/master/doc/man7/migration_guide.pod

I'd love to contribute, but I'll likely only have time if this ever becomes a major blocker for me.

[juchem]

The least I can do is provide a minimal Dockerfile with a repro of the issue above: Dockerfile.txt

[edenhill]

Merged

[KalleOlaviNiemitalo]

OpenSSL 3.0.x versions are not under the same license terms as OpenSSL 1.1.1n. That change may cause some extra work for application vendors who redistribute binaries from librdkafka.redist.