kafka-connect.service fails with new java version

[amonsef82]

We faced the below issue appears to be due to the ciphers deprecated in the new upgraded java version from 1.8.0.181 to 1.8.0_242 and it looks like a problem in new version cryptography.

Kafka version is kafka-1.1.0-confluent5.3.1 Hadoop version is 2.6.5

trace": "org.apache.kafka.connect.errors.ConnectException: java.io.IOException: Login failure for [email protected] from keytab /home/kafka/.keytabs/cedp-prod/kafkaflow.keytab: javax.security.auth.login.LoginException: Message stream modified (41)\n\tat io.confluent.connect.hdfs.DataWriter.\u003cinit\u003e(DataWriter.java:346)\n\tat io.confluent.connect.hdfs.DataWriter.\u003cinit\u003e(DataWriter.java:102)\n\tat io.confluent.connect.hdfs.HdfsSinkTask.start(HdfsSinkTask.java:88)\n\tat org.apache.kafka.connect.runtime.WorkerSinkTask.initializeAndStart(WorkerSinkTask.java:300)\n\tat org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:189)\n\tat org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:177)\n\tat org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:227)\n\tat java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)\n\tat java.util.concurrent.FutureTask.run(FutureTask.java:266)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)\n\tat java.lang.Thread.run(Thread.java:748)\nCaused by: java.io.IOException: Login failure for [email protected] from keytab /home/kafka/.keytabs/cedp-prod/kafkaflow.keytab: javax.security.auth.login.LoginException: Message stream modified (41)\n\tat org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:963)\n\tat io.confluent.connect.hdfs.DataWriter.\u003cinit\u003e(DataWriter.java:173)\n\t... 11 more\nCaused by: javax.security.auth.login.LoginException: Message stream modified (41)\n\tat com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:808)\n\tat com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:618)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:498)\n\tat javax.security.auth.login.LoginContext.invoke(LoginContext.java:755)\n\tat javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)\n\tat javax.security.auth.login.LoginContext$4.run(LoginContext.java:682)\n\tat javax.security.auth.login.LoginContext$4.run(LoginContext.java:680)\n\tat java.security.AccessController.doPrivileged(Native Method)\n\tat javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)\n\tat javax.security.auth.login.LoginContext.login(LoginContext.java:587)\n\tat org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:954)\n\t... 12 more\nCaused by: KrbException: Message stream modified (41)\n\tat sun.security.krb5.KrbKdcRep.check(KrbKdcRep.java:101)\n\tat sun.security.krb5.KrbAsRep.decrypt(KrbAsRep.java:159)\n\tat sun.security.krb5.KrbAsRep.decryptUsingKeyTab(KrbAsRep.java:121)\n\tat sun.security.krb5.KrbAsReqBuilder.resolve(KrbAsReqBuilder.java:308)\n\tat sun.security.krb5.KrbAsReqBuilder.action(KrbAsReqBuilder.java:447)\n\tat trace": "org.apache.kafka.connect.errors.ConnectException: java.io.IOException: Login failure for [email protected] from keytab /home/kafka/.keytabs/cedp-prod/kafkaflow.keytab: javax.security.auth.login.LoginException: Message stream modified (41)\n\tat io.confluent.connect.hdfs.DataWriter.\u003cinit\u003e(DataWriter.java:346)\n\tat io.confluent.connect.hdfs.DataWriter.\u003cinit\u003e(DataWriter.java:102)\n\tat io.confluent.connect.hdfs.HdfsSinkTask.start(HdfsSinkTask.java:88)\n\tat org.apache.kafka.connect.runtime.WorkerSinkTask.initializeAndStart(WorkerSinkTask.java:300)\n\tat org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:189)\n\tat org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:177)\n\tat org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:227)\n\tat java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)\n\tat java.util.concurrent.FutureTask.run(FutureTask.java:266)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)\n\tat java.lang.Thread.run(Thread.java:748)\nCaused by: java.io.IOException: Login failure for [email protected] from keytab /home/kafka/.keytabs/cedp-prod/kafkaflow.keytab: javax.security.auth.login.LoginException: Message stream modified (41)\n\tat org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:963)\n\tat io.confluent.connect.hdfs.DataWriter.\u003cinit\u003e(DataWriter.java:173)\n\t... 11 more\nCaused by: javax.security.auth.login.LoginException: Message stream modified (41)\n\tat com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:808)\n\tat com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:618)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:498)\n\tat javax.security.auth.login.LoginContext.invoke(LoginContext.java:755)\n\tat javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)\n\tat javax.security.auth.login.LoginContext$4.run(LoginContext.java:682)\n\tat javax.security.auth.login.LoginContext$4.run(LoginContext.java:680)\n\tat java.security.AccessController.doPrivileged(Native Method)\n\tat javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)\n\tat javax.security.auth.login.LoginContext.login(LoginContext.java:587)\n\tat org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:954)\n\t... 12 more\nCaused by: KrbException: Message stream modified (41)\n\tat sun.security.krb5.KrbKdcRep.check(KrbKdcRep.java:101)\n\tat sun.security.krb5.KrbAsRep.decrypt(KrbAsRep.java:159)\n\tat sun.security.krb5.KrbAsRep.decryptUsingKeyTab(KrbAsRep.java:121)\n\tat sun.security.krb5.KrbAsReqBuilder.resolve(KrbAsReqBuilder.java:308)\n\tat sun.security.krb5.KrbAsReqBuilder.action(KrbAsReqBuilder.java:447)\n\tat

[amonsef82]

Any help?

[OneCricketeer]

You're getting a Kerberos login failure... That's not an issue with this connector, rather your certificates or your hadoop.conf dir files.

You can use hadoop fs and kinit to debug your problem, but again, not something that should be solved by this project