cruise-control icon indicating copy to clipboard operation
cruise-control copied to clipboard

Connecting SASL_SSL Kafka Cluster

Open tommyJimmy87 opened this issue 6 years ago • 28 comments

I'm using Security Protocol set to "SASL_SSL" which means I need to specify all the JKS and JAAS in order to connect to the Cluster. I'm providing those information through the property file, but I found out that when the KafkaAdminClient runs it gets wrong SSL configuration because of this :

// Configure SSL configs (if security protocol is SSL)
      if (securityProtocol.equals(SecurityProtocol.SSL.name)) {
        setStringConfigIfExists(configs, adminClientConfigs, SslConfigs.SSL_TRUSTMANAGER_ALGORITHM_CONFIG);
        setStringConfigIfExists(configs, adminClientConfigs, SslConfigs.SSL_KEYMANAGER_ALGORITHM_CONFIG);
        setStringConfigIfExists(configs, adminClientConfigs, SslConfigs.SSL_KEYSTORE_TYPE_CONFIG);
        setStringConfigIfExists(configs, adminClientConfigs, SslConfigs.SSL_KEYSTORE_LOCATION_CONFIG);
        setStringConfigIfExists(configs, adminClientConfigs, SslConfigs.SSL_TRUSTSTORE_TYPE_CONFIG);
        setStringConfigIfExists(configs, adminClientConfigs, SslConfigs.SSL_TRUSTSTORE_LOCATION_CONFIG);
        setStringConfigIfExists(configs, adminClientConfigs, SslConfigs.SSL_SECURE_RANDOM_IMPLEMENTATION_CONFIG);
        setPasswordConfigIfExists(configs, adminClientConfigs, SslConfigs.SSL_KEYSTORE_PASSWORD_CONFIG);
        setPasswordConfigIfExists(configs, adminClientConfigs, SslConfigs.SSL_KEY_PASSWORD_CONFIG);
        setPasswordConfigIfExists(configs, adminClientConfigs, SslConfigs.SSL_TRUSTSTORE_PASSWORD_CONFIG);
      }

I'm not using SSL but SASL_SSL and I still need those properties to be set in the KafkaClientAdmin configurations.

The class where i found that code portion is KafkaCruiseControlUtils.java.

This is the error I'm facing :

org.apache.kafka.common.KafkaException: Failed create new KafkaAdminClient
	at org.apache.kafka.clients.admin.KafkaAdminClient.createInternal(KafkaAdminClient.java:370)
	at org.apache.kafka.clients.admin.AdminClient.create(AdminClient.java:62)
	at com.linkedin.kafka.cruisecontrol.KafkaCruiseControlUtils.createAdminClient(KafkaCruiseControlUtils.java:176)
	at com.linkedin.kafka.cruisecontrol.KafkaCruiseControlUtils.describeLogDirs(KafkaCruiseControlUtils.java:161)
	at com.linkedin.kafka.cruisecontrol.servlet.response.KafkaClusterState.populateKafkaBrokerLogDirState(KafkaClusterState.java:323)
	at com.linkedin.kafka.cruisecontrol.servlet.response.KafkaClusterState.getJsonStructure(KafkaClusterState.java:252)
	at com.linkedin.kafka.cruisecontrol.servlet.response.KafkaClusterState.getJSONString(KafkaClusterState.java:85)
	at com.linkedin.kafka.cruisecontrol.servlet.response.KafkaClusterState.discardIrrelevantAndCacheRelevant(KafkaClusterState.java:454)
	at com.linkedin.kafka.cruisecontrol.servlet.response.AbstractCruiseControlResponse.discardIrrelevantResponse(AbstractCruiseControlResponse.java:43)
	at com.linkedin.kafka.cruisecontrol.servlet.response.AbstractCruiseControlResponse.writeSuccessResponse(AbstractCruiseControlResponse.java:34)
	at com.linkedin.kafka.cruisecontrol.servlet.KafkaCruiseControlServletUtils.syncRequest(KafkaCruiseControlServletUtils.java:118)
	at com.linkedin.kafka.cruisecontrol.servlet.KafkaCruiseControlServlet.doGet(KafkaCruiseControlServlet.java:159)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:841)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:535)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
	at org.eclipse.jetty.server.Server.handle(Server.java:564)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:317)
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:110)
	at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
	at org.eclipse.jetty.util.thread.Invocable.invokePreferred(Invocable.java:128)
	at org.eclipse.jetty.util.thread.Invocable$InvocableExecutor.invoke(Invocable.java:222)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:294)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:199)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:673)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:591)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.kafka.common.KafkaException: java.lang.IllegalArgumentException: No serviceName defined in either JAAS or Kafka config
	at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:153)
	at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:140)
	at org.apache.kafka.common.network.ChannelBuilders.clientChannelBuilder(ChannelBuilders.java:65)
	at org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:88)
	at org.apache.kafka.clients.admin.KafkaAdminClient.createInternal(KafkaAdminClient.java:346)
	... 39 more
Caused by: java.lang.IllegalArgumentException: No serviceName defined in either JAAS or Kafka config
	at org.apache.kafka.common.security.kerberos.KerberosLogin.getServiceName(KerberosLogin.java:303)
	at org.apache.kafka.common.security.kerberos.KerberosLogin.configure(KerberosLogin.java:92)
	at org.apache.kafka.common.security.authenticator.LoginManager.<init>(LoginManager.java:63)
	at org.apache.kafka.common.security.authenticator.LoginManager.acquireLoginManager(LoginManager.java:114)
	at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:142)
	... 43 more

Tag Cruise Control version is 2.0.36. Kafka Version is 1.1.x.

tommyJimmy87 avatar Mar 06 '19 17:03 tommyJimmy87

@tommyJimmy87 Thanks for reporting the issue! Cruise Control does not support SASL_SSL, yet -- i.e. it currently supports SSL and Plaintext. Would you be interested in submitting a patch to add SASL_SSL support?

efeg avatar Mar 07 '19 02:03 efeg

@tommyJimmy87 Are you trying to use Kerberos for authentication? If so, one of the following configs is needed:

  1. sasl.kerberos.service.name
  2. sasl.jaas.config

It would be more helpful if you can share the configurations you are using.

becketqin avatar Mar 07 '19 07:03 becketqin

@becketqin We are not using kerberos... we are just using SASL_SSL with the username and password in the JAAS.. but apparently as @efeg said is not supported.

tommyJimmy87 avatar Mar 07 '19 09:03 tommyJimmy87

@efeg I can take a look how complicate is to patch it.

tommyJimmy87 avatar Mar 07 '19 09:03 tommyJimmy87

@tommyJimmy87 Hmm, CC should support all the mechanisms support by Kafka itself. Have you set sasl.mechanism in your config? By default it is going to be GSSAPI and kerberos will be used. With SASL_SSL, change the that config to PLAIN and have a try.

becketqin avatar Mar 07 '19 09:03 becketqin

Also, please make sure you are using the org.apache.kafka.common.security.plain.PlainLoginModule with your username and password in the jaas config file.

becketqin avatar Mar 07 '19 09:03 becketqin

@becketqin Thx, here my config file :

broker.metric.sample.store.topic=__KafkaCruiseControlModelTrainingSamples
sample.store.class=com.linkedin.kafka.cruisecontrol.monitor.sampling.KafkaSampleStore
ssl.key.password=xxxxxxxxx (password is present)
min.samples.per.broker.metrics.window=1
num.partition.metrics.windows=1
ssl.keystore.password=xxxxxxxxx (password is present)
sample.store.topic.replication.factor=2
ssl.keystore.location=/etc/kafka/secrets/jks/kafka.keystore.jks
capacity.config.file=/etc/kafka-cruise-control/config/capacity.json
webserver.ui.diskpath=/app/cruise-control-ui/dist/
zookeeper.connect=ZKIP:2181/devkafka
partition.metrics.window.ms=300000
metric.sampler.partition.assignor.class=com.linkedin.kafka.cruisecontrol.monitor.sampling.DefaultMetricSamplerPartitionAssignor
ssl.truststore.location=/etc/kafka/secrets/jks/kafka.truststore.jks
ssl.client.auth=true
partition.metric.sample.store.topic=__KafkaCruiseControlPartitionMetricSamples
sasl.mechanism=PLAIN
security.protocol=SASL_SSL
producer.security.protocol=SASL_SSL
client.id=kafka-cruise-control
broker.metrics.window.ms=300000
default.goals=com.linkedin.kafka.cruisecontrol.analyzer.goals.RackAwareGoal,com.linkedin.kafka.cruisecontrol.analyzer.goals.ReplicaCapacityGoal,com.linkedin.kafka.cruisecontrol.analyzer.goals.DiskCapacityGoal,com.linkedin.kafka.cruisecontrol.analyzer.goals.NetworkInboundCapacityGoal,com.linkedin.kafka.cruisecontrol.analyzer.goals.NetworkOutboundCapacityGoal,com.linkedin.kafka.cruisecontrol.analyzer.goals.CpuCapacityGoal,com.linkedin.kafka.cruisecontrol.analyzer.goals.ReplicaDistributionGoal,com.linkedin.kafka.cruisecontrol.analyzer.goals.PotentialNwOutGoal,com.linkedin.kafka.cruisecontrol.analyzer.goals.DiskUsageDistributionGoal,com.linkedin.kafka.cruisecontrol.analyzer.goals.NetworkInboundUsageDistributionGoal,com.linkedin.kafka.cruisecontrol.analyzer.goals.NetworkOutboundUsageDistributionGoal,com.linkedin.kafka.cruisecontrol.analyzer.goals.CpuUsageDistributionGoal,com.linkedin.kafka.cruisecontrol.analyzer.goals.TopicReplicaDistributionGoal,com.linkedin.kafka.cruisecontrol.analyzer.goals.LeaderBytesInDistributionGoal
producer.sasl.mechanism=PLAIN
bootstrap.servers=SASL_SSL://myIP:9094
cluster.configs.file=/etc/kafka-cruise-control/config/clusterConfigs.json
num.sample.loading.threads=8
num.broker.metrics.windows=20
ssl.truststore.password=xxxxxxxxx (password is present)
ssl.endpoint.identification.algorithm=
metric.sampler.class=com.linkedin.kafka.cruisecontrol.monitor.sampling.CruiseControlMetricsReporterSampler
consumer.sasl.mechanism=PLAIN
consumer.security.protocol=SASL_SSL
num.metric.fetcher=1
auto.create.topics.enable=true
metric.sampling.interval.ms=120000
metric.reporter.topic.pattern=__CruiseControlMetrics

And also I'm providing my JAAS file path in the KAFKA_OPTS env variable that should be added when CC runs. We are using the same JAAS file for several application.

tommyJimmy87 avatar Mar 07 '19 09:03 tommyJimmy87

@becketqin The problem is just the KafkaAdminClient Configs, the ConsumerConf for example are created with the right values (jks.. password and jaas are present in the ConsumerConf but not in the KafkaAdminClient).

tommyJimmy87 avatar Mar 07 '19 09:03 tommyJimmy87

@becketqin Uhm.. with the tag 0.1.39 is working actually.. earlier I was using 2.0.36 Tag... what's the different ? Master branch should be at the 0.1.39 right ? I guess it was a little bit confusing.

tommyJimmy87 avatar Mar 07 '19 11:03 tommyJimmy87

@tommyJimmy87 Thanks for the update. This sounds like a bug in 2.0.36. But I could be wrong. @efeg would be more familiar with the diff between those two versions.

becketqin avatar Mar 07 '19 11:03 becketqin

@tommyJimmy87 The reason why the tag 0.1.39 is working, but 2.0.36 is not is because former versions (i.e. 0.1.*) do not call describeLogDirs to show logDirs in the response of kafka_cluster_state (i.e. the endpoint that your request is sent to). The describeLogDirs creates and uses a KafkaAdminClient to return the corresponding response (supported for Kafka: 1.1.0+ with Cruise Control: 2.*).

To support SASL_SSL in 2.*, I believe there are (at least) two places to update (and test) -- i.e. you may search for the string if (securityProtocol.equals(SecurityProtocol.SSL.name)) to find the exact locations:

  1. KafkaCruiseControlUtils#parseAdminClientConfigs
  2. CruiseControlMetricsUtils#addSslConfigs(Properties, CruiseControlMetricsReporterConfig).

^ So I suspect that all we need would be:

else if (securityProtocol.equals(SecurityProtocol.SASL_SSL.name)) {
  setStringConfigIfExists(configs, adminClientConfigs, SaslConfigs.SASL_JAAS_CONFIG);
  setPasswordConfigIfExists(configs, adminClientConfigs, SaslConfigs.*); // Needs update for any other relevant password configs
  setStringConfigIfExists(configs, adminClientConfigs, SaslConfigs.*); // Needs update for any other relevant string configs.
}

efeg avatar Mar 07 '19 17:03 efeg

@efeg Ok as I thought. If you want I can contribute on this.

tommyJimmy87 avatar Mar 08 '19 13:03 tommyJimmy87

@tommyJimmy87 Sure, we appreciate contributions!

efeg avatar Mar 08 '19 16:03 efeg

I've encountered the same issue with SASL_PLAINTEXT and sasl.mechanism=SCRAM-SHA-256: Cruise Control does connects to the cluster but right after that it tries to start AdminClient which does not inherit client.id, jaas.config and sasl.mechanism

UPD: managed to fix this by adding import org.apache.kafka.common.config.SaslConfigs; and setStringConfigIfExists(configs, adminClientConfigs, SaslConfigs.SASL_MECHANISM); into KafkaCruiseControlUtils.java i guess the same should be added into CruiseControlMetricsUtils.java

Sugaroverdose avatar Apr 05 '19 13:04 Sugaroverdose

@Sugaroverdose Sounds great! Would you consider creating a PR with the patch?

efeg avatar Apr 05 '19 17:04 efeg

@efeg i'll try to it seems like that CruiseControlMetricsUtils.java does not inherit securityProtocol, so idk how it works with something different than plaintext

Sugaroverdose avatar Apr 05 '19 20:04 Sugaroverdose

@tommyJimmy87 you may try latest migrate_to_kafka_2_0 branch, it should work in your case now

Sugaroverdose avatar Apr 09 '19 10:04 Sugaroverdose

I'm novice in adopting to cruise-contol using 2.029,trying with Sasl_ssl going by above discussion it looks I'm may not be able use it sasl_ssl could you suggest which release shld I pick .. Thank u

poyyi avatar May 02 '19 14:05 poyyi

@poyyi What is your Cruise Control Metric Jar configuration?

tommyJimmy87 avatar May 03 '19 07:05 tommyJimmy87

@tommy it is cruise-control-metrics-reporter-0.1.0-SNAPSHOT.jar META-INF/ META-INF/MANIFEST.MF com/ com/linkedin/ com/linkedin/kafka/ com/linkedin/kafka/cruisecontrol/ com/linkedin/kafka/cruisecontrol/metricsreporter/ com/linkedin/kafka/cruisecontrol/metricsreporter/CruiseControlMetricsReporter$1.class com/linkedin/kafka/cruisecontrol/metricsreporter/CruiseControlMetricsReporter.class com/linkedin/kafka/cruisecontrol/metricsreporter/metric/ com/linkedin/kafka/cruisecontrol/metricsreporter/metric/YammerMetricProcessor$Context.class com/linkedin/kafka/cruisecontrol/metricsreporter/metric/YammerMetricProcessor.class com/linkedin/kafka/cruisecontrol/metricsreporter/metric/CruiseControlMetric$MetricClassId.class com/linkedin/kafka/cruisecontrol/metricsreporter/metric/CruiseControlMetric.class com/linkedin/kafka/cruisecontrol/metricsreporter/metric/RawMetricType$MetricScope.class com/linkedin/kafka/cruisecontrol/metricsreporter/metric/RawMetricType.class com/linkedin/kafka/cruisecontrol/metricsreporter/metric/BrokerMetric.class com/linkedin/kafka/cruisecontrol/metricsreporter/metric/MetricSerde$1.class com/linkedin/kafka/cruisecontrol/metricsreporter/metric/MetricSerde.class com/linkedin/kafka/cruisecontrol/metricsreporter/metric/MetricsUtils.class com/linkedin/kafka/cruisecontrol/metricsreporter/metric/PartitionMetric.class com/linkedin/kafka/cruisecontrol/metricsreporter/metric/TopicMetric.class com/linkedin/kafka/cruisecontrol/metricsreporter/CruiseControlMetricsReporterConfig.class com/linkedin/kafka/cruisecontrol/metricsreporter/exception/

on the kafka servers.. "2024" is sasl_ssl port metric.reporters=com.linkedin.kafka.cruisecontrol.metricsreporter.CruiseControlMetricsReporter cruise.control.metrics.reporter.bootstrap.servers=0.0.0.0:2024 auto.create.topics.enable=true

CC property file bootstrap.servers=BK1-IP:2024,BK2-IP:2024,BK3-IP:2024,BK4-IP:2024,BK5-IP:2024 zookeeper.connect=ZK1-IP:2181,ZK2-IP:2181,ZK3-IP:2181 sample.store.class=com.linkedin.kafka.cruisecontrol.monitor.sampling.KafkaSampleStore ssl.key.password=XXXXX@123 ssl.keystore.password=XXXXX@123 ssl.keystore.location=/opt/app/work/kafka-0.10.01/certs/kafka.server.truststore.jks ssl.truststore.location=/opt/app/work/kafka-0.10.01/certs/kafka.server.truststore.jks ssl.client.auth=true sasl.mechanism=PLAIN security.protocol=SASL_SSL producer.security.protocol=SASL_SSL producer.sasl.mechanism=PLAIN ssl.truststore.password=XXXXX@123 ssl.endpoint.identification.algorithm= consumer.sasl.mechanism=PLAIN consumer.security.protocol=SASL_SSL

poyyi avatar May 04 '19 23:05 poyyi

I guess you are missing all the Cruise Control configs for Sasl SSL.

Try with these also :


KAFKA_METRIC_REPORTERS: "com.linkedin.kafka.cruisecontrol.metricsreporter.CruiseControlMetricsReporter"
  KAFKA_CRUISE_CONTROL_METRICS_REPORTER_SECURITY_PROTOCOL: "SASL_SSL"
  KAFKA_CRUISE_CONTROL_METRICS_REPORTER_SASL_MECHANISM: "PLAIN"

  KAFKA_CRUISE_CONTROL_METRICS_REPORTER_SSL_ENDPOINT_IDENTIFICATION_ALGORITHM: ""
  KAFKA_CRUISE_CONTROL_METRICS_REPORTER_SSL_CLIENT_AUTH: "true"
  KAFKA_CRUISE_CONTROL_METRICS_REPORTER_SSL_KEYSTORE_LOCATION: '/path/to/kafka.server.keystore.jks'
  KAFKA_CRUISE_CONTROL_METRICS_REPORTER_SSL_TRUSTSTORE_LOCATION: '/path/to/kafka.server.truststore.jks'

  KAFKA_CRUISE_CONTROL_METRICS_REPORTER_SASL_MECHANISM_INTER_BROKER_PROTOCOL: "{{.kafkaSaslMechanismInterBrokerProtocol}}"
  KAFKA_CRUISE_CONTROL_METRICS_REPORTER_SASL_ENABLED_MECHANISMS: "{{.kafkaSaslEnabledMechanisms}}"

  KAFKA_CRUISE_CONTROL_METRICS_REPORTER_LISTENER_SECURITY_PROTOCOL_MAP: "INTERNAL:SASL_SSL,CLIENT:SASL_SSL,BROKER:SASL_SSL,PLAINTEXT:PLAINTEXT,SSL:SSL,SASL_SSL:SASL_SSL"

  KAFKA_CRUISE_CONTROL_METRICS_REPORTER_LISTENER_SECURITY_PROTOCOL_MAP: "INTERNAL:SSL,CLIENT:SSL,BROKER:SSL,PLAINTEXT:PLAINTEXT,SSL:SSL"

  KAFKA_CRUISE_CONTROL_METRICS_REPORTER_LISTENER_SECURITY_PROTOCOL_MAP: "INTERNAL:PLAINTEXT,CLIENT:PLAINTEXT,BROKER:PLAINTEXT,PLAINTEXT:PLAINTEXT,SSL:SSL"
 KAFKA_CRUISE_CONTROL_METRICS_REPORTER_ADVERTISED_LISTENERS={advertised_listeners_list}

KAFKA_CRUISE_CONTROL_METRICS_REPORTER_SSL_TRUSTSTORE_PASSWORD={trustStorePass}
KAFKA_CRUISE_CONTROL_METRICS_REPORTER_SSL_KEYSTORE_PASSWORD={keyStorePass}
KAFKA_CRUISE_CONTROL_METRICS_REPORTER_SSL_KEY_PASSWORD={keyPass}
KAFKA_CRUISE_CONTROL_METRICS_REPORTER_BOOTSTRAP_SERVERS={yoursBootstrapServer}

tommyJimmy87 avatar May 05 '19 08:05 tommyJimmy87

@jimmy sorry could not try immly on SASL_SSL (but tired with SSL which i run into some other issue described below ).. i have few queries on your suggestions for SASL_SSL

1)Above mentioned properties shld go into server.properties on brokers? 2) shld they be given exactly like how you gave above KAFKA_METRIC_REPORTERS: ?, currently i have it as below in my server.properties metric.reporters=com.linkedin.kafka.cruisecontrol.metricsreporter.CruiseControlMetricsReporter cruise.control.metrics.reporter.security.protocol=SSL

PL note version I'm using: Cruise control version: Cruise-control-2.0.42 Kafka :kafka_2.12-0.11.0.0 (Kafka version : 0.11.0.1)

I ran with SSL pl find my setting and Exception I'm getting for

Failures "GET /kafkacruisecontrol/kafka_cluster_state?json=true HTTP/1.1" 500 4462 79

Success "GET /kafkacruisecontrol/user_tasks?json=true HTTP/1.1" 200 2637 8 "GET /kafkacruisecontrol/state?substates=EXECUTOR&verbose=true&json=true HTTP/1.1" 200 61 2 "GET /kafkacruisecontrol/state?substates=MONITOR&verbose=true&json=true HTTP/1.1" 200 288 1 "GET /kafkacruisecontrol/state?substates=ANALYZER&verbose=true&json=true HTTP/1.1" 200 2866 2 "GET /kafkacruisecontrol/load?allow_capacity_estimation=true&json=true HTTP/1.1" 200 3074 2 "GET /kafkacruisecontrol/user_tasks?json=true HTTP/1.1" 200 3075 2 "GET /kafkacruisecontrol/state?substates=EXECUTOR&verbose=true&json=true HTTP/1.1" 200 61 1 "GET /kafkacruisecontrol/user_tasks?json=true HTTP/1.1" 200 3275 2

Exception: GET request '/kafka_cluster_state' due to: 'Failed to populate broker logDir state.'. (com.linkedin.kafka.cruisecontrol.servlet.KafkaCruiseControlServlet) java.lang.RuntimeException: Failed to populate broker logDir state. at com.linkedin.kafka.cruisecontrol.servlet.response.KafkaClusterState.getJSONString(KafkaClusterState.java:90) at com.linkedin.kafka.cruisecontrol.servlet.response.KafkaClusterState.discardIrrelevantAndCacheRelevant(KafkaClusterState.java:456) at com.linkedin.kafka.cruisecontrol.servlet.response.AbstractCruiseControlResponse.discardIrrelevantResponse(AbstractCruiseControlResponse.java:49) Caused by: org.apache.kafka.common.errors.UnsupportedVersionException: The broker does not support DESCRIBE_LOG_DIRS ---------------Cruisecontrol.properties Setting---- Cruise control.properties security.protocol=SSL ssl.truststore.type=JKS ssl.truststore.location=/opt/app/workload/cruise-control-2.0.42/config/certs/kafka.server.truststore.jks ssl.truststore.password=pxxxxxxxx@123 ssl.enabled.protocols=TLSv1.2,TLSv1.1,TLSv1 #sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="uxxxxxx" password="pxxxxxxxxx";

---------Broker server.properties---------------- listeners=PLAINTEXT://bk1:2021,SASL_PLAINTEXT://bk1:2022,SSL://bk1:2023,SASL_SSL://bk1:2024 ssl.enabled.protocols=TLSv1.2,TLSv1.1,TLSv1 security.inter.broker.protocol=SASL_SSL sasl.mechanism.inter.broker.protocol=PLAIN sasl.enabled.mechanisms=PLAIN super.users=User:mxxxx allow.everyone.if.no.acl.found=true authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer

advertised.listeners=PLAINTEXT://bk1:2021,SASL_PLAINTEXT://bk1:2022,SSL://bk1:2023,SASL_SSL://bk1:2024 ssl.enabled.protocols=TLSv1.2,TLSv1.1,TLSv1

############################# Log Basics #############################

log.dirs=/opt/app/workload/enabler/kafka_2.12-0.11.0.0/kafka-logs

#####Cruise control setting############## group.initial.rebalance.delay.ms=0 metric.reporters=com.linkedin.kafka.cruisecontrol.metricsreporter.CruiseControlMetricsReporter cruise.control.metrics.reporter.security.protocol=SSL cruise.control.metrics.reporter.bootstrap.servers=0.0.0.0:2023 auto.create.topics.enable=true cruise.control.metrics.reporter.ssl.truststore.location=/opt/app/workload/kafka_2.12-0.11.0.0/config/certs/kafka.server.truststore.jks cruise.control.metrics.reporter.ssl.truststore.password=pxxxxxxxx@123 cruise.control.metrics.reporter.ssl.keystore.location=/opt/app/workload/kafka_2.12-0.11.0.0/config/certs/kafka.server.keystore.jks cruise.control.metrics.reporter.ssl.keystore.password=pxxxxxxxx@123 cruise.control.metrics.reporter.ssl.key.password=pxxxxxxxx@123 broker.rack=Rack1

poyyi avatar May 17 '19 23:05 poyyi

@tommyJimmy87 ...did you get chance to look at my query

poyyi avatar May 24 '19 19:05 poyyi

I am facing the same issue. Can you please update on this ?

ishita07-danaher avatar Dec 04 '19 08:12 ishita07-danaher

Does cruise control support TLS SASL/Scram enabled Kafka secure cluster?

Arullaldivakar avatar Mar 30 '21 14:03 Arullaldivakar