librdkafka icon indicating copy to clipboard operation
librdkafka copied to clipboard
verified publisher icon confluentinc

Cyrus/libsasl2 is missing a GSSAPI module

Open SolaTian opened this issue 1 year ago • 8 comments

Read the FAQ first: https://github.com/confluentinc/librdkafka/wiki/FAQ

Do NOT create issues for questions, use the discussion forum: https://github.com/confluentinc/librdkafka/discussions

Description

%7|1716569607.172|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://11.82.37.28:21007/bootstrap]: sasl_plaintext://11.82.37.28:21007/bootstrap: Feature IdempotentProducer: InitProducerId (0..0) supported by broker %7|1716569607.172|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://11.82.37.28:21007/bootstrap]: sasl_plaintext://11.82.37.28:21007/bootstrap: Enabling feature IdempotentProducer %7|1716569607.172|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://11.82.37.28:21007/bootstrap]: sasl_plaintext://11.82.37.28:21007/bootstrap: Feature ZSTD: Produce (7..7) NOT supported by broker %7|1716569607.172|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://11.82.37.28:21007/bootstrap]: sasl_plaintext://11.82.37.28:21007/bootstrap: Feature ZSTD: Fetch (10..10) NOT supported by broker %7|1716569607.172|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://11.82.37.28:21007/bootstrap]: sasl_plaintext://11.82.37.28:21007/bootstrap: Disabling feature ZSTD %7|1716569607.172|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://11.82.37.28:21007/bootstrap]: sasl_plaintext://11.82.37.28:21007/bootstrap: Feature SaslAuthReq: SaslHandshake (1..1) supported by broker %7|1716569607.172|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://11.82.37.28:21007/bootstrap]: sasl_plaintext://11.82.37.28:21007/bootstrap: Feature SaslAuthReq: SaslAuthenticate (0..1) supported by broker %7|1716569607.172|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://11.82.37.28:21007/bootstrap]: sasl_plaintext://11.82.37.28:21007/bootstrap: Enabling feature SaslAuthReq %7|1716569607.172|FEATURE|rdkafka#producer-1| [thrd:sasl_plaintext://11.82.37.28:21007/bootstrap]: sasl_plaintext://11.82.37.28:21007/bootstrap: Updated enabled protocol features to MsgVer1,ApiVersion,BrokerBalancedConsumer,ThrottleTime,Sasl,SaslHandshake,BrokerGroupCoordinator,LZ4,OffsetTime,MsgVer2,IdempotentProducer,SaslAuthReq %7|1716569607.172|AUTH|rdkafka#producer-1| [thrd:sasl_plaintext://11.82.37.28:21007/bootstrap]: sasl_plaintext://11.82.37.28:21007/bootstrap: Auth in state APIVERSION_QUERY (handshake supported) %7|1716569607.172|STATE|rdkafka#producer-1| [thrd:sasl_plaintext://11.82.37.28:21007/bootstrap]: sasl_plaintext://11.82.37.28:21007/bootstrap: Broker changed state APIVERSION_QUERY -> AUTH_HANDSHAKE %7|1716569607.172|BROADCAST|rdkafka#producer-1| [thrd:sasl_plaintext://11.82.37.28:21007/bootstrap]: Broadcasting state change %7|1716569607.172|SEND|rdkafka#producer-1| [thrd:sasl_plaintext://11.82.37.28:21007/bootstrap]: sasl_plaintext://11.82.37.28:21007/bootstrap: Sent SaslHandshakeRequest (v1, 29 bytes @ 0, CorrId 3) %7|1716569607.177|RECV|rdkafka#producer-1| [thrd:sasl_plaintext://11.82.37.28:21007/bootstrap]: sasl_plaintext://11.82.37.28:21007/bootstrap: Received SaslHandshakeResponse (v1, 14 bytes, CorrId 3, rtt 5.23ms) %7|1716569607.177|SASLMECHS|rdkafka#producer-1| [thrd:sasl_plaintext://11.82.37.28:21007/bootstrap]: sasl_plaintext://11.82.37.28:21007/bootstrap: Broker supported SASL mechanisms: GSSAPI %7|1716569607.177|AUTH|rdkafka#producer-1| [thrd:sasl_plaintext://11.82.37.28:21007/bootstrap]: sasl_plaintext://11.82.37.28:21007/bootstrap: Auth in state AUTH_HANDSHAKE (handshake supported) %7|1716569607.177|STATE|rdkafka#producer-1| [thrd:sasl_plaintext://11.82.37.28:21007/bootstrap]: sasl_plaintext://11.82.37.28:21007/bootstrap: Broker changed state AUTH_HANDSHAKE -> AUTH_REQ %7|1716569607.177|BROADCAST|rdkafka#producer-1| [thrd:sasl_plaintext://11.82.37.28:21007/bootstrap]: Broadcasting state change %7|1716569607.177|SASL|rdkafka#producer-1| [thrd:sasl_plaintext://11.82.37.28:21007/bootstrap]: sasl_plaintext://11.82.37.28:21007/bootstrap: Initializing SASL client: service name kafka, hostname 11.82.37.28, mechanisms GSSAPI, provider Cyrus %7|1716569607.178|SASL|rdkafka#producer-1| [thrd:sasl_plaintext://11.82.37.28:21007/bootstrap]: sasl_plaintext://11.82.37.28:21007/bootstrap: My supported SASL mechanisms: EXTERNAL %2|1716569607.178|LIBSASL|rdkafka#producer-1| [thrd:sasl_plaintext://11.82.37.28:21007/bootstrap]: sasl_plaintext://11.82.37.28:21007/bootstrap: Cyrus/libsasl2 is missing a GSSAPI module: make sure the libsasl2-modules-gssapi-mit or cyrus-sasl-gssapi packages are installed %7|1716569607.178|FAIL|rdkafka#producer-1| [thrd:sasl_plaintext://11.82.37.28:21007/bootstrap]: sasl_plaintext://11.82.37.28:21007/bootstrap: Failed to initialize SASL authentication: SASL handshake failed (start (-4)): SASL(-4): no mechanism available: No worthy mechs found (after 0ms in state AUTH_REQ) (_AUTHENTICATION) %3|1716569607.178|FAIL|rdkafka#producer-1| [thrd:sasl_plaintext://11.82.37.28:21007/bootstrap]: sasl_plaintext://11.82.37.28:21007/bootstrap: Failed to initialize SASL authentication: SASL handshake failed (start (-4)): SASL(-4): no mechanism available: No worthy mechs found (after 0ms in state AUTH_REQ)

How to reproduce

I configured the Kerberos with the option --with-gss_impl=mit --enable-plain --enable-gssapi --with-dblib=no --without-des --without-saslauthd (cyrus-sasl-2.1.27),but when I try to get Authentication, it indicate that My supported SASL mechanisms: EXTERNAL，Cyrus/libsasl2 is missing a GSSAPI module.why's that?

Checklist

IMPORTANT: We will close issues where the checklist has not been completed.

Please provide the following information:

  • [x] librdkafka version (release number or git tag): <librdkafka-2.3.0>
  • [x] Apache Kafka version: <2.3.0>
  • [x] librdkafka client configuration: <message.max.bytes = 8388608; debug = generic,broker,topic,metadata,feature,queue,msg,protocol,cgrp,security,fetch,interceptor,plugin,consumer,admin,eos,mock,assignor,conf,all; socket.timeout.ms = 5000; api.version.request = true; security.protocol = sasl_plaintext; sasl.mechanisms = GSSAPI; sasl.kerberos.service.name = Kerberos_Service_Name; sasl.kerberos.principal = Kerberos_Principal; sasl.kerberos.kinit.cmd = kinit -k -t "%{sasl.kerberos.keytab}" %{sasl.kerberos.principal}; sasl.kerberos.keytab = /etc/user.keytab; queue.buffering.max.messages = 3; queue.buffering.max.ms = 10;>
  • [x] Operating system: <Ubuntu>
  • [ ] Provide logs ( debug = generic,broker,topic,metadata,feature,queue,msg,protocol,cgrp,security,fetch,interceptor,plugin,consumer,admin,eos,mock,assignor,conf,all ) from librdkafka
  • [x] Provide broker log excerpts
  • [x] Critical issue

SolaTian avatar May 27 '24 01:05 SolaTian

Hi @SolaTian have you installed cyrus-sasl-gssapi in client machine too?

emasab avatar May 29 '24 11:05 emasab

Hi @SolaTian have you installed cyrus-sasl-gssapi in client machine too?

I'm really sorry, I don't quite understand what you said about installing cyrus-sasl-gssapi on the client machine. Do you mean that I need to do additional operations besides cross compiling the cyrus-sasl library and linking it to librdkafka? Is cyrus-sasl-gssapi a tool generated after cross compiling cyrus-sasl?

SolaTian avatar May 30 '24 03:05 SolaTian

Side question. Seems that confluent shipped 2.4.0 deb's have been compiled without gssapi support. 2.3.0 still has it. Is that intended?

AudriusButkevicius avatar Jun 04 '24 00:06 AudriusButkevicius

Given there was a pipeline migration, 2.4.0 version of Debian packages was compiled without libsasl2 support, it's fixed now in deb version 2.4.0-3

emasab avatar Jun 04 '24 07:06 emasab

Thanks for the clarification, and sorry for hijacking the thread.

AudriusButkevicius avatar Jun 04 '24 07:06 AudriusButkevicius

@SolaTian about 2.3.0: cyrus-sasl-gssapi is a plugin for the GSSAPI SASL mechanism that is dynamically loaded so you have to install the rpm package https://rpmfind.net/linux/rpm2html/search.php?query=cyrus-sasl-gssapi(x86-64)

emasab avatar Jun 04 '24 12:06 emasab

@SolaTian about 2.3.0: cyrus-sasl-gssapi is a plugin for the GSSAPI SASL mechanism that is dynamically loaded so you have to install the rpm package https://rpmfind.net/linux/rpm2html/search.php?query=cyrus-sasl-gssapi(x86-64)

@emasab Thank you very much. I cross compiled the cyrus sasl2.1.27 library. Is the plugin name generated in the cross compilation environment libgssapiv2. soor some other dynamic libraries? And I had already linked the static library libgssapiv2. a generated by cross compilation, but still reported an error that does not support GSSAPI. Is it necessary to load the dynamic library libgssapiv2. so on the client machine

SolaTian avatar Jun 05 '24 03:06 SolaTian

Is it necessary to load the dynamic library libgssapiv2. so on the client machine

Exactly the .so is dynamically loaded by libsasl2

emasab avatar Jun 06 '24 08:06 emasab

I'm experiencing the same issue after upgrading librdkafka. In the previous version (v2.2.0), I was able to successfully link against the static library libgssapiv2.a without any problems. However, after upgrading to librdkafka v2.10, I'm now encountering this GSSAPI module missing error.

Could you clarify if there were any changes to SASL2/GSSAPI support between these versions?

Any guidance on the proper configuration for v2.10 would be greatly appreciated.

litao3rd avatar Jun 20 '25 04:06 litao3rd