SASL Support

[webe3]

I would like to use pykafka to connect to IBM Bluemix Message Hub with is a Kafka implementation. There is a kafka-python example in this thread. https://stackoverflow.com/questions/35294949/can-i-call-the-bluemix-message-hub-service-from-python/40325181#40325181

I also found this for confluent-kafka:

A new Python client

Confluent, the company founded by the creators of Kafka, has released a Python client (https://github.com/confluentinc/confluent-kafka-python). It is built as a thin bindings layer on top of librdkafka and works for both Python 2 and 3.

To use with Message Hub, simply pass these properties in when creating a Producer/Consumer:

'ssl.ca.location': '/etc/ssl/certs/',
'sasl.mechanisms': 'PLAIN',
'sasl.username': '<USER>',
'sasl.password': '<PASSWORD>',
'security.protocol': 'sasl_ssl'

Note: The value of ‘ssl.ca.location depends’ on your operating system. For example:

Ubuntu: /etc/ssl/certs/
RedHat: /etc/pki/tls/cert.pem
OS X: select system root certificates from Keychain Access and export as .pem on the filesystem.

Is there a way to connect using pykafka? If so, how would I do it?

[emmettbutler]

Hi @webe3, thanks for reaching out. Sounds like you're looking for pykafka's SSL support, which is based on constructing an SslConfig object and passing it to a newly created KafkaClient.

[webe3]

Yes, I saw that but it is not clear to me how to do it given what I have to work with.

[webe3]

It doesn't look to me like pykafka has the mechanism for sasl plain like kafka-python and confluent_kafka do. For example in kafka-python I would pass in the following configurations to create a consumer or producer.

configs = { 'bootstrap_servers': credentials['kafka_brokers_sasl'], 'sasl_mechanism': 'PLAIN', 'sasl_plain_username': credentials['api_key'][0:16], 'sasl_plain_password': credentials['api_key'][16:48], 'security_protocol': 'SASL_SSL', 'ssl_context': ssl.create_default_context(), }

These parameters are described here: https://kafka-python.readthedocs.io/en/master/apidoc/KafkaConsumer.html

I don't see any similar parameters for sasl for pykafka.

[emmettbutler]

Thanks for clarifying, @webe3. Pykafka indeed doesn't have SASL support, only TLS/SSL support at the moment. I've changed the title of this bug to indicate the need for SASL support. If you're interested in contributing a pull request for this feature, I'd be happy to review it. Otherwise, it will go into the issue backlog.

[Battleroid]

SASL support would be great. We're having to rewrite an internal project to use kafka-python for the time being due to the lack of SASL support.

[jayanra]

Do we have SASL support enabled for PyKafka?

[emmettbutler]

No, SASL support is still pending a pull request that adds it.

[ZamElek]

Our kafka prod server is moving to GSSAPI SASL (kerberos) authentication, so I would like to know are there any news regarding SASL support by pykafka?

[zhangjiahaol]

我们的产品将更换抛弃pykafka，因为支持SASK，求支持~

[colin404]

in our products we use kafka and give up pykafka due to the lack of sasl support. it will be great if pykafka support sasl

[ares2233]

SASL is required in our project. pykafka is not support.