No way to authenticate Kafka Clients (SSL / SASL)

[amit-k-yadav]

I am trying to enable simple SASL for clients (It is okay if Brokers and Zookeepers talk to each other in PLAINTEXT as all of our traffic will always be internal) following this guide but I am failing to get this working.

I have modified the Dockerfile (FOR NOW) to have the required files mentioned in Confluent's documentation. The error I am getting is:

....
eperConnectionWatcher@79fc0f2f
[main] INFO org.apache.zookeeper.common.X509Util - Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation
[main] INFO org.apache.zookeeper.ClientCnxnSocket - jute.maxbuffer value is 4194304 Bytes
[main] INFO org.apache.zookeeper.ClientCnxn - zookeeper.request.timeout value is 0. feature enabled=
[main-SendThread(confluent-kafka-1-cp-zookeeper-headless:2181)] WARN org.apache.zookeeper.ClientCnxn - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/opt/kafka/data-0/kafka_client_jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if
Zookeeper server allows it.
[main-SendThread(confluent-kafka-1-cp-zookeeper-headless:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server confluent-kafka-1-cp-zookeeper-headless/10.60.1.61:2181
[main] ERROR io.confluent.admin.utils.ClusterStatus - Error occurred while connecting to Zookeeper server[confluent-kafka-1-cp-zookeeper-headless:2181]. Authentication failed.
[main-SendThread(confluent-kafka-1-cp-zookeeper-headless:2181)] INFO org.apache.zookeeper.ClientCnxn - Socket connection established, initiating session, client: /10.60.1.60:34350, server: confluent-kafka-1-cp-zookeeper-headless/10.60.1.61:2181
[main-SendThread(confluent-kafka-1-cp-zookeeper-headless:2181)] INFO org.apache.zookeeper.ClientCnxn - Session establishment complete on server confluent-kafka-1-cp-zookeeper-headless/10.60.1.61:2181, sessionid = 0x1000ed674dd0005, negotiated timeout = 40000
[main] INFO org.apache.zookeeper.ZooKeeper - Session: 0x1000ed674dd0005 closed
[main-EventThread] INFO org.apache.zookeeper.ClientCnxn - EventThread shut down for session: 0x1000ed674dd0005

My configuration in values.yaml looks like:

...
## Kafka Server properties
## ref: https://kafka.apache.org/documentation/#configuration
configurationOverrides:
  "offsets.topic.replication.factor": "1"
  "default.replication.factor": 1
  "opts": "-Djava.security.auth.login.config=/opt/kafka/data-0/kafka_server_jaas.conf -Djava.security.auth.login.config=/opt/kafka/data-0/kafka_client_jaas.conf"
  # "min.insync.replicas": 2
  # "auto.create.topics.enable": false

  ## Options required for external access via NodePort
  ## ref:
  ## - http://kafka.apache.org/documentation/#security_configbroker
  ## - https://cwiki.apache.org/confluence/display/KAFKA/KIP-103%3A+Separation+of+Internal+and+External+traffic
  ##
  ## Advertised listeners will use the firstListenerPort value as it's default unless overridden here.
  ## Setting "advertised.listeners" here appends to "PLAINTEXT://${POD_IP}:9092,"
  "zookeeper.sasl.client": false
  "ZOOKEEPER_SET_ACL": false
  "listeners": "SASL_PLAINTEXT://0.0.0.0:9093"
  "security.inter.broker.protocol": "PLAINTEXT"
  "sasl.mechanism.inter.broker.protocol": "PLAIN"
  "sasl.enabled.mechanisms": "PLAIN"
  "advertised.listeners": |-
   SASL_PLAINTEXT://0.0.0.0:9093
  "listener.security.protocol.map": |-
   PLAINTEXT:PLAINTEXT,EXTERNAL:PLAINTEXT

File kafka_client_jaas.conf has the below content:

KafkaClient {
  org.apache.kafka.common.security.plain.PlainLoginModule required
  username="alice"
  password="alice-secret";
};

File kafka_server_jaas.conf has:

KafkaServer {
   org.apache.kafka.common.security.plain.PlainLoginModule required
   username="admin"
   password="admin-secret"
   user_admin="admin-secret"
   user_alice="alice-secret";
};

Is it not possible with this chart?

[leshibily]

Hello folks,

Is anyone tracking this issue? Any work going on currently?

[adv4000]

Current version of Helm Chart not supporting this, you will need to modify it:

1. Create k8s secret file with JAAS config for KafkaServer/KafkaClient/Client/Server

apiVersion: v1
kind: Secret
metadata:
  name: my-jaas-config
type: Opaque
data:
  kafka_jaas.conf:  |-
    Ly8gRm9yIEthZmthIC0gZG353453453453453rZXJzIGNvbW11bmljYXRlIGJldHdlZW4gdGhl
    bSwgYW5kIGNyZWF0ZSBw345345345345345345345345345LYWZrYVNlcnZlciB7CiAgIG9y

2. Add customEnv parameter into Zookeepr Chart same as you already exist in Kafka Chart, modify statefulset.yaml in zookeeper to use this, just copy/paste from kafka
3. Provide KAFKA_OPTS to CustomEnv into both: Kafka and Zookeepr charts:

customEnv:
  KAFKA_OPTS: "-Djava.security.auth.login.config=/etc/kafka/secrets/kafka_jaas.conf"

4. Now you need to modify statefulset.yaml in both: Kafka and Zookeeper to mount Secret as volume to /etc/kafka/secrets/

      volumes:
      - name: jaas-config
        secret:
          secretName: my-jaas-config
          
          
        volumeMounts:
          - name: jaas-config
            mountPath: /etc/kafka/secrets
            

Hope this will help, good luck !

[Sukhdk]

Hi @amit-k-yadav were you able to fix this issue?

[yurykomarov]

Up

[vjvel]

Can we use TLS to encrypt the connectivity between kafka clients and brokers ? we are having scenario need to expose the kafka server and whitelist the Ip but also we need to enable encryption? @adv4000