Kafka + kerbos authentication +GSSAPI for node js application -> broker transport failure

[dileep006]

Read the FAQ first: https://github.com/edenhill/librdkafka/wiki/FAQ

Do NOT create issues for questions, use the discussion forum: https://github.com/edenhill/librdkafka/discussions

Description

We are trying to use Kafka + kerbos authentication +GSSAPI for node js application for enterprise chat applictaion. We are getting below error . What could be the reason for this error , any help is highly appreciated . if you have any referennce for similar implemntaion , please share ...

broker transport failure.

It is working fine for JAVA client , failing from NODE JS Client.

this is our client configurations. const kafkaConsumer = new KafkaConsumer({ 'client.id': ['myserver', hostname()].join('-'), 'group.id': 'kf__write_test', 'rebalance_cb': true, 'metadata.broker.list': [servers], 'security.protocol': "sasl_ssl", 'sasl.mechanisms': 'GSSAPI', 'sasl.password': 'changeit', 'ssl.ca.location': path.resolve(__dirname, '../../cache/certs/...'), "sasl.kerberos.service.name": "", "sasl.kerberos.principal": ", "sasl.kerberos.keytab": path.resolve(__dirname, '../../cache/certs/..'), "sasl.kerberos.kinit.cmd": "%{sasl.kerberos.keytab}", "debug": "all" }, { 'auto.offset.reset': 'beginning', })

How to reproduce

Above is the configuration

IMPORTANT: Always try to reproduce the issue on the latest released version (see https://github.com/edenhill/librdkafka/releases), if it can't be reproduced on the latest version the issue has been fixed.

Checklist

IMPORTANT: We will close issues where the checklist has not been completed.

Please provide the following information:

• [x] librdkafka version (release number or git tag): 2.14.0
• [ ] Apache Kafka version: ``
• [x] librdkafka client configuration: Mentioned above
• [ ] Operating system: trying on my loacl macbook
• [ ] Provide logs (with debug=.. as necessary) from librdkafka
• [ ] Provide broker log excerpts
• [ ] Critical issue

[edenhill]

Reproduce with "debug": "protocol,security,broker" and provide the logs.

[edenhill]

Please also specify the librdkafka version in use, with debugging enabled it will be emitted soon after the client instance is created.

[dileep006]

protocol,security,broker

please see the error.

[dileep006]

Please also specify the librdkafka version in use, with debugging enabled it will be emitted soon after the client instance is created.

"node-rdkafka": "^2.14.0",

[dileep006]

Do you have sample client code for Kafka + kerbos authentication +GSSAPI for node js ? just want to make sure i was follwing the right configuration

[edenhill]

"node-rdkafka": "^2.14.0",

That's the node-rdkafka version, not librdkafka version

[edenhill]

Reproduce with "debug": "protocol,security,broker" and provide the logs.

[dileep006]

"node-rdkafka": "^2.14.0",

That's the node-rdkafka version, not librdkafka version

1.9.2

[dileep006]

Reproduce with "debug": "protocol,security,broker" and provide the logs.

2022-12-19T-debug.log

320 timing command:install Completed in 41507ms 321 verbose stack Error: command failed 321 verbose stack at ChildProcess. (/Users//.nvm/versions/node/v16.10.0/lib/node_modules/npm/node_modules/@npmcli/promise-spawn/index.js:64:27) 321 verbose stack at ChildProcess.emit (node:events:390:28) 321 verbose stack at maybeClose (node:internal/child_process:1064:16) 321 verbose stack at Process.ChildProcess._handle.onexit (node:internal/child_process:301:5) 322 verbose pkgid [email protected] 323 verbose cwd /Users//APP/APP/chat-socket-server 324 verbose Darwin 21.6.0 325 verbose argv "/Users//.nvm/versions/node/v16.10.0/bin/node" "/Users//.nvm/versions/node/v16.10.0/bin/npm" "install" "debug" 326 verbose node v16.10.0 327 verbose npm v7.24.0 328 error code 1 329 error path /Users//APP/APP/chat-socket-server/node_modules/node-rdkafka 330 error command failed 331 error command sh -c node-gyp rebuild 332 error Debugger attached. 332 error gyp info it worked if it ends with ok 332 error gyp info using [email protected] 332 error gyp info using [email protected] | darwin | x64 332 error gyp info find Python using Python version 3.10.9 found at "/Library/Frameworks/Python.framework/Versions/3.10/bin/python3" 332 error gyp info spawn /Library/Frameworks/Python.framework/Versions/3.10/bin/python3 332 error gyp info spawn args [ 332 error gyp info spawn args '/Users//APP/APP/chat-socket-server/node_modules/node-rdkafka/node_modules/node-gyp/gyp/gyp_main.py', 332 error gyp info spawn args 'binding.gyp', 332 error gyp info spawn args '-f', 332 error gyp info spawn args 'make', 332 error gyp info spawn args '-I', 332 error gyp info spawn args '/Users//APP/APP/chat-socket-server/node_modules/node-rdkafka/build/config.gypi', 332 error gyp info spawn args '-I', 332 error gyp info spawn args '/Users//APP/APP/chat-socket-server/node_modules/node-rdkafka/node_modules/node-gyp/addon.gypi', 332 error gyp info spawn args '-I', 332 error gyp info spawn args '/Users//Library/Caches/node-gyp/16.10.0/include/node/common.gypi', 332 error gyp info spawn args '-Dlibrary=shared_library', 332 error gyp info spawn args '-Dvisibility=default', 332 error gyp info spawn args '-Dnode_root_dir=/Users//Library/Caches/node-gyp/16.10.0', 332 error gyp info spawn args '-Dnode_gyp_dir=/Users//APP/APP/chat-socket-server/node_modules/node-rdkafka/node_modules/node-gyp', 332 error gyp info spawn args '-Dnode_lib_file=/Users//Library/Caches/node-gyp/16.10.0/<(target_arch)/node.lib', 332 error gyp info spawn args '-Dmodule_root_dir=/Users//APP/APP/chat-socket-server/node_modules/node-rdkafka', 332 error gyp info spawn args '-Dnode_engine=v8', 332 error gyp info spawn args '--depth=.', 332 error gyp info spawn args '--no-parallel', 332 error gyp info spawn args '--generator-output', 332 error gyp info spawn args 'build', 332 error gyp info spawn args '-Goutput_dir=.' 332 error gyp info spawn args ] 332 error Debugger attached. 332 error Waiting for the debugger to disconnect... 332 error gyp: Call to 'node ./util/get-env.js BUILD_LIBRDKAFKA 1' returned exit status 0 while in binding.gyp. while trying to load binding.gyp 332 error gyp ERR! configure error 332 error gyp ERR! stack Error: gyp failed with exit code: 1 332 error gyp ERR! stack at ChildProcess.onCpExit (/Users//APP/APP/chat-socket-server/node_modules/node-rdkafka/node_modules/node-gyp/lib/configure.js:259:16) 332 error gyp ERR! stack at ChildProcess.emit (node:events:390:28) 332 error gyp ERR! stack at Process.ChildProcess._handle.onexit (node:internal/child_process:290:12) 332 error gyp ERR! System Darwin 21.6.0 332 error gyp ERR! command "/Users//.nvm/versions/node/v16.10.0/bin/node" "/Users//APP/APP/chat-socket-server/node_modules/node-rdkafka/node_modules/.bin/node-gyp" "rebuild" 332 error gyp ERR! cwd /Users//APP/APP/chat-socket-server/node_modules/node-rdkafka 332 error gyp ERR! node -v v16.10.0 332 error gyp ERR! node-gyp -v v8.4.1 332 error gyp ERR! not ok 332 error Waiting for the debugger to disconnect... 333 verbose exit 1

[dileep006]

Also , i am using node 16.10.0 .Please suggest the right node version also .

[dileep006]

adding a note SASL with plain text is workign fine with node-rdkafka , SASL with GSSAPI is failing with broker transport failure

const kafkaConsumer = new KafkaConsumer({ 'client.id': ['myserver', hostname()].join('-'), 'group.id': 'kf_ts_sh_cvdshp_write_test', 'rebalance_cb': true, 'metadata.broker.list': '', 'security.protocol': "sasl_plaintext", 'sasl.mechanisms': 'PLAIN', 'sasl.password': ' 'sasl.username': '', "debug": "protocol,security,broker" }, { 'auto.offset.reset': 'beginning', })

[dileep006]

@edenhill , any help is highly appreciated ?

[aasitvora99]

Hello, were you able to solve this?