kafka-connect-hdfs icon indicating copy to clipboard operation
kafka-connect-hdfs copied to clipboard

Multiple keytab kerberos issue

Open zeun0725 opened this issue 2 years ago • 4 comments

Problem

If multiple key tabs are distributed on the connect server, authentication errors may occur.

      UserGroupInformation.loginUserFromKeytab(principal, connectorConfig.connectHdfsKeytab()); // problem point
      final UserGroupInformation ugi = UserGroupInformation.getLoginUser();  // problem point
      log.info("Login as: " + ugi.getUserName());

      isRunning = true;
      ticketRenewThread = new Thread(() -> renewKerberosTicket(ugi));

example) A connector uses the test user keytab (write path: /hdfs/user/test), and B connector uses the test2 user keytab (write path: /hdfs/user/test2). If both connectors are restarted due to certain circumstances, different key tab information can be imported with thread-based behavior. If the B connector calls loginUserFromKeytab('[email protected]' , '/../test2.keytab') when the A connector calls getLoginUser(), A login as test2 user. So you can face the error as below. Caused by: org.apache.ranger.authorization.hadoop.exceptions.RangerAccessControlException: Permission denied: user=test2, access=EXECUTE, inode="/hdfs/user/test"

Solution

final UserGroupInformation ugi = UserGroupInformation.loginUserFromKeytabAndReturnUGI(principal, connectorConfig.connectHdfsKeytab());

When receiving an ugi instance, it was changed to authenticate with a keytab and receive it. Change to use loginUserFromKeytabAndReturnUGI() func.

Does this solution apply anywhere else?
  • [ ] yes
  • [ X ] no
If yes, where?

Test Strategy

Testing done:
  • [ X ] Unit tests
  • [ ] Integration tests
  • [ ] System tests
  • [ X ] Manual tests

Release Plan

zeun0725 avatar Nov 25 '23 05:11 zeun0725

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


박지은 seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

cla-assistant[bot] avatar Nov 25 '23 05:11 cla-assistant[bot]

@kkonstantine @snehashisp Could you please review this PR? Related to this issue https://github.com/confluentinc/kafka-connect-hdfs/issues/325

@zeun0725 Please sign the cla and fix checkstyle issue to make this valid pr.

lala7573 avatar Dec 20 '23 07:12 lala7573

@zeun0725 I build this and tested it. It seems that there are other problems besides this issue. This changes only affect on relogin. When I try to create a connector with different user, lease permission error still occurs.

lala7573 avatar Dec 21 '23 08:12 lala7573

@lala7573 Can you share the test code or config value?

zeun0725 avatar Mar 05 '24 01:03 zeun0725