CMAK
CMAK copied to clipboard
Login module not specified in JAAS config - kafka-manager-1.3.3.21
Hi
I am running Kafka with SASL_PLAINTEXT. I have configed Kafka-manager to use this and provided a jaas file with the correct options (well options that work for all other consumers) - but when starting kafka-manager I get the error "Login module not specified in JAAS config"
The consumers & offsets are not visible in the UI.
version kafka-manager-1.3.3.21
jaas file:
KafkaClient {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
keyTab="/etc/security/keytabs/kafka.service.keytab"
storeKey=true
useTicketCache=false
serviceName="kafka"
principal="XXXXXXX";
};
2018-10-29 22:20:20,650 - [ERROR] - from kafka.manager.actor.cluster.KafkaStateActor in kafka-manager-system-akka.actor.default-dispatcher-2
KafkaTopicOffsetGetter exception
org.apache.kafka.common.KafkaException: Failed to construct kafka consumer
at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:793) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:644) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:624) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
at kafka.manager.actor.cluster.KafkaStateActor$KafkaTopicOffsetGetter$$anonfun$run$7$$anonfun$apply$10$$anonfun$apply$mcV$sp$14.apply(KafkaStateActor.scala:1490) ~[kafka-manager.kafka-manager-1.3.3.21-sans-externalized.jar:na]
at kafka.manager.actor.cluster.KafkaStateActor$KafkaTopicOffsetGetter$$anonfun$run$7$$anonfun$apply$10$$anonfun$apply$mcV$sp$14.apply(KafkaStateActor.scala:1468) ~[kafka-manager.kafka-manager-1.3.3.21-sans-externalized.jar:na]
at scala.collection.Iterator$class.foreach(Iterator.scala:891) ~[org.scala-lang.scala-library-2.11.12.jar:na]
at scala.collection.AbstractIterator.foreach(Iterator.scala:1334) ~[org.scala-lang.scala-library-2.11.12.jar:na]
at scala.collection.MapLike$DefaultKeySet.foreach(MapLike.scala:174) ~[org.scala-lang.scala-library-2.11.12.jar:na]
at kafka.manager.actor.cluster.KafkaStateActor$KafkaTopicOffsetGetter$$anonfun$run$7$$anonfun$apply$10.apply$mcV$sp(KafkaStateActor.scala:1468) ~[kafka-manager.kafka-manager-1.3.3.21-sans-externalized.jar:na]
at scala.util.control.Breaks.breakable(Breaks.scala:38) ~[org.scala-lang.scala-library-2.11.12.jar:na]
at kafka.manager.actor.cluster.KafkaStateActor$KafkaTopicOffsetGetter$$anonfun$run$7.apply(KafkaStateActor.scala:1467) ~[kafka-manager.kafka-manager-1.3.3.21-sans-externalized.jar:na]
at kafka.manager.actor.cluster.KafkaStateActor$KafkaTopicOffsetGetter$$anonfun$run$7.apply(KafkaStateActor.scala:1436) ~[kafka-manager.kafka-manager-1.3.3.21-sans-externalized.jar:na]
at scala.Option.fold(Option.scala:158) ~[org.scala-lang.scala-library-2.11.12.jar:na]
at kafka.manager.actor.cluster.KafkaStateActor$KafkaTopicOffsetGetter.run(KafkaStateActor.scala:1436) ~[kafka-manager.kafka-manager-1.3.3.21-sans-externalized.jar:na]
at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_191]
Caused by: java.lang.IllegalArgumentException: Login module not specified in JAAS config
at org.apache.kafka.common.security.JaasConfig.<init>(JaasConfig.java:66) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
at org.apache.kafka.common.security.JaasContext.load(JaasContext.java:90) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
at org.apache.kafka.common.security.JaasContext.loadClientContext(JaasContext.java:84) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:119) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
at org.apache.kafka.common.network.ChannelBuilders.clientChannelBuilder(ChannelBuilders.java:65) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
at org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:88) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:710) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
... 14 common frames omitted
Same problem here, any solution?
Same problem with last release 1.3.3.22
are u specifying the jaas config in the kafka manger cluster config?
I think so and it is the same issue as #591
Are you not sure where you specify the JAAS config? There 2 places. One is through conf file as command line argument when starting kafka manager. The other is through the cluster config inside of kafka manger where you create/modify the cluster config.
@patelh same problem with last release 1.3.3.22. And when I create the cluster config using kafka manager through web UI, I'm confused with the SASL JAAS Config, should I fill with the path to the config file or the content of the file?
And another question, I have multiple Kafka Cluster (each with a zookeeper cluster using different SASL), how could I specify the SASL information about the zookeeper cluster so I can use manager to create/delete topic ? I think the first way you mentioned above just for the zookeeper cluster where manager to store the meta data not for the broker's zookeeper cluster.
Using kafka manager UI, it is better but not perfect. Consumers are not displayed I have these message
2019-01-15 14:08:28,364 - [INFO] - from org.apache.zookeeper.ClientCnxn in kafka-manager-system-akka.actor.default-dispatcher-2-SendThread(114.221.241.80:2181)
Opening socket connection to server 114.221.241.80/114.221.241.80:2181. Will not attempt to authenticate using SASL (unknown error)
and this messsage:
2019-01-15 14:08:30,161 - [ERROR] - from kafka.manager.actor.cluster.package$TryLogErrorHelper in KafkaManagedOffsetCache
Failed to create consumer for offset topic for cluster KAFKA_BPCE-IT_INT_ant
org.apache.kafka.common.KafkaException: Failed to construct kafka consumer
at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:793) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:644) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:624) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
at kafka.manager.actor.cluster.KafkaManagedOffsetCache.kafka$manager$actor$cluster$KafkaManagedOffsetCache$$createKafkaConsumer(KafkaStateActor.scala:283) ~[kafka-manager.kafka-manager-1.3.3.22-sans-externalized.jar:na]
at kafka.manager.actor.cluster.KafkaManagedOffsetCache$$anonfun$run$2.apply(KafkaStateActor.scala:327) ~[kafka-manager.kafka-manager-1.3.3.22-sans-externalized.jar:na]
followed by
2019-01-15 14:08:30,178 - [ERROR] - from kafka.manager.actor.cluster.KafkaStateActor in kafka-manager-system-akka.actor.default-dispatcher-4
consumerProperties:{key.deserializer=org.apache.kafka.common.serialization.ByteArrayDeserializer, sasl.mechanism=SCRAM-SHA-512, bootstrap.servers=a-bilibgk002.dom101.intres:9093, group.id=kafka.manager.actor.cluster.KafkaStateActor.KafkaTopicOffsetGetter, sasl.jaas.config=/appli/kafka/kafka-manager-1.3.3.22/conf/jaas.conf, value.deserializer=org.apache.kafka.common.serialization.ByteArrayDeserializer, security.protocol=SASL_PLAINTEXT} WARNING arguments left: 1
2019-01-15 14:08:30,178 - [ERROR] - from kafka.manager.actor.cluster.KafkaStateActor in kafka-manager-system-akka.actor.default-dispatcher-4
KafkaTopicOffsetGetter exception
org.apache.kafka.common.KafkaException: Failed to construct kafka consumer
at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:793) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:644) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:624) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
at kafka.manager.actor.cluster.KafkaStateActor$KafkaTopicOffsetGetter$$anonfun$run$7$$anonfun$apply$10$$anonfun$apply$mcV$sp$14.apply(KafkaStateActor.scala:1490) ~[kafka-manager.kafka-manager-1.3.3.22-sans-externalized.jar:na]
at kafka.manager.actor.cluster.KafkaStateActor$KafkaTopicOffsetGetter$$anonfun$run$7$$anonfun$apply$10$$anonfun$apply$mcV$sp$14.apply(KafkaStateActor.scala:1468) ~[kafka-manager.kafka-manager-1.3.3.22-sans-externalized.jar:na]
at scala.collection.Iterator$class.foreach(Iterator.scala:891) ~[org.scala-lang.scala-library-2.11.12.jar:na]
at scala.collection.AbstractIterator.foreach(Iterator.scala:1334) ~[org.scala-lang.scala-library-2.11.12.jar:na]
at scala.collection.MapLike$DefaultKeySet.foreach(MapLike.scala:174) ~[org.scala-lang.scala-library-2.11.12.jar:na]
at kafka.manager.actor.cluster.KafkaStateActor$KafkaTopicOffsetGetter$$anonfun$run$7$$anonfun$apply$10.apply$mcV$sp(KafkaStateActor.scala:1468) ~[kafka-manager.kafka-manager-1.3.3.22-sans-externalized.jar:na]
at scala.util.control.Breaks.breakable(Breaks.scala:38) ~[org.scala-lang.scala-library-2.11.12.jar:na]
at kafka.manager.actor.cluster.KafkaStateActor$KafkaTopicOffsetGetter$$anonfun$run$7.apply(KafkaStateActor.scala:1467) ~[kafka-manager.kafka-manager-1.3.3.22-sans-externalized.jar:na]
followd by
2019-01-15 14:08:40,259 - [ERROR] - from kafka.manager.actor.cluster.KafkaStateActor in kafka-manager-system-akka.actor.default-dispatcher-2
KafkaTopicOffsetGetter exception
org.apache.kafka.common.KafkaException: Failed to construct kafka consumer
at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:793) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:644) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:624) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
at kafka.manager.actor.cluster.KafkaStateActor$KafkaTopicOffsetGetter$$anonfun$run$7$$anonfun$apply$10$$anonfun$apply$mcV$sp$14.apply(KafkaStateActor.scala:1490) ~[kafka-manager.kafka-manager-1.3.3.22-sans-externalized.jar:na]
at kafka.manager.actor.cluster.KafkaStateActor$KafkaTopicOffsetGetter$$anonfun$run$7$$anonfun$apply$10$$anonfun$apply$mcV$sp$14.apply(KafkaStateActor.scala:1468) ~[kafka-manager.kafka-manager-1.3.3.22-sans-externalized.jar:na]
at scala.collection.Iterator$class.foreach(Iterator.scala:891) ~[org.scala-lang.scala-library-2.11.12.jar:na]
at scala.collection.AbstractIterator.foreach(Iterator.scala:1334) ~[org.scala-lang.scala-library-2.11.12.jar:na]
at scala.collection.MapLike$DefaultKeySet.foreach(MapLike.scala:174) ~[org.scala-lang.scala-library-2.11.12.jar:na]
at kafka.manager.actor.cluster.KafkaStateActor$KafkaTopicOffsetGetter$$anonfun$run$7$$anonfun$apply$10.apply$mcV$sp(KafkaStateActor.scala:1468) ~[kafka-manager.kafka-manager-1.3.3.22-sans-externalized.jar:na]
at scala.util.control.Breaks.breakable(Breaks.scala:38) ~[org.scala-lang.scala-library-2.11.12.jar:na]
at kafka.manager.actor.cluster.KafkaStateActor$KafkaTopicOffsetGetter$$anonfun$run$7.apply(KafkaStateActor.scala:1467) ~[kafka-manager.kafka-manager-1.3.3.22-sans-externalized.jar:na]
at kafka.manager.actor.cluster.KafkaStateActor$KafkaTopicOffsetGetter$$anonfun$run$7.apply(KafkaStateActor.scala:1436) ~[kafka-manager.kafka-manager-1.3.3.22-sans-externalized.jar:na]
at scala.Option.fold(Option.scala:158) ~[org.scala-lang.scala-library-2.11.12.jar:na]
at kafka.manager.actor.cluster.KafkaStateActor$KafkaTopicOffsetGetter.run(KafkaStateActor.scala:1436) ~[kafka-manager.kafka-manager-1.3.3.22-sans-externalized.jar:na]
at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_181]
Caused by: java.lang.IllegalArgumentException: Login module not specified in JAAS config
at org.apache.kafka.common.security.JaasConfig.<init>(JaasConfig.java:66) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
at org.apache.kafka.common.security.JaasContext.load(JaasContext.java:90) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
at org.apache.kafka.common.security.JaasContext.loadClientContext(JaasContext.java:84) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:119) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
at org.apache.kafka.common.network.ChannelBuilders.clientChannelBuilder(ChannelBuilders.java:65) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
at org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:88) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:710) ~[org.apache.kafka.kafka-clients-1.1.0.jar:na]
... 14 common frames omitted
2019-01-15 14:08:45,283 - [ERROR] - from kafka.manager.actor.cluster.KafkaStateActor in kafka-manager-system-akka.actor.default-dispatcher-2
consumerProperties:{key.deserializer=org.apache.kafka.common.serialization.ByteArrayDeserializer, sasl.mechanism=SCRAM-SHA-512, bootstrap.servers=a-bilibgk002.dom101.intres:9093, group.id=kafka.manager.actor.cluster.KafkaStateActor.KafkaTopicOffsetGetter, sasl.jaas.config=/appli/kafka/kafka-manager-1.3.3.22/conf/jaas.conf, value.deserializer=org.apache.kafka.common.serialization.ByteArrayDeserializer, security.protocol=SASL_PLAINTEXT} WARNING arguments left: 1
2019-01-15 14:08:45,284 - [ERROR] - from kafka.manager.actor.cluster.KafkaStateActor in kafka-manager-system-akka.actor.default-dispatcher-2
KafkaTopicOffsetGetter exception
Hi,
I was having the same problem, but I managed to resolve the error, at least in my config.
Basically the JAAS Config from the UI expect a JAAS file content (KafkaClient Section), not a path to the jaas file.
yes, it seems better :-) now, it is a problem with invalid credentials
[warn] o.a.k.c.NetworkClient - [Consumer clientId=consumer-1, groupId=KMOffsetCache-bilibgk007.dom101.intres] Connection to node -1 terminated during authentication. This may indicate that authentication failed due to invalid credentials.
@doddys thanks a lot, and I have a question that the create/delete topic feature is ok ?
doddys [email protected] 于2019年2月4日周一 下午3:34写道:
Hi,
I was having the same problem, but I managed to resolve the error, at least in my config.
Basically the JAAS Config from the UI expect a JAAS file content (KafkaClient Section), not a path to the jaas file.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/yahoo/kafka-manager/issues/577#issuecomment-460154170, or mute the thread https://github.com/notifications/unsubscribe-auth/AEI0tzb5FxCO5ahCcb3-kH2veUGEr_2vks5vJ-JtgaJpZM4YAspY .
Hi,
I was having the same problem, but I managed to resolve the error, at least in my config.
Basically the JAAS Config from the UI expect a JAAS file content (KafkaClient Section), not a path to the jaas file.
Yeah, i also struggled on this, finally it is working, even i was passing the jaas file path, after changing to content it started working. Thanks
@doddys thanks a lot, and I have a question that the create/delete topic feature is ok ? doddys [email protected] 于2019年2月4日周一 下午3:34写道: …
Yes I am able to create and delete topic. I am using kafka user keytab.
@doddys I'm getting the same error. Can you please share your jaas.conf file?
@maldo91 I use content likes below, seems work, no error in application log org.apache.kafka.common.security.plain.PlainLoginModule required username="xxx" password="xxxx";
@yantzu Are you OK with the list of consumers ?
good news (at least for me), it is working :-) Great, thanks for the job
I ran in to the same issue as OP and finally got this working by entering the sasl.jaas.config value directly in the UI when creating a cluster. Has anybody else noticed that Kafka Manager then keeps printing out the SASL password in plaintext to the console repeatedly?
2019-09-23 15:44:21,573 - [INFO] k.m.a.c.KafkaStateActor - SASL Mechanism =SASL_MECHANISM_SCRAM512
2019-09-23 15:44:21,573 - [INFO] k.m.a.c.KafkaStateActor - SASL JAAS config=org.apache.kafka.common.security.scram.ScramLoginModule required username="superuser" password="***";
Any way to avoid this?
@doddys thanks a lot, and I have a question that the create/delete topic feature is ok ? doddys [email protected] 于2019年2月4日周一 下午3:34写道: …
Yes I am able to create and delete topic. I am using kafka user keytab. hi doddys ,how to confiig JAAS file content for user keytab ?
Hi, guys, I solved this problem by login to zookeeper by zkCli.sh and delete our kafka-manager
config stored in zookeeper. I misconfigured the Java JAAS Config when I add a new cluster in GUI(by setting it to a path, yet it requires just a string java JAAS config, not a file containing it), so I cannot restart the CMAK, so I delete the config for the cluster in zookeeper, and restart CMAK, and add the cluster again. Now it's all fine.
Hope this help.
Hi, I was having the same problem, but I managed to resolve the error, at least in my config. Basically the JAAS Config from the UI expect a JAAS file content (KafkaClient Section), not a path to the jaas file.
Yeah, i also struggled on this, finally it is working, even i was passing the jaas file path, after changing to content it started working. Thanks can you share you "SASL JAAS Config (only applies to SASL based security)" on UI ?
it is not working for me, can any one please provide complete steps?