Bug in zookeerper kerberos auth

[tianchao-haohan]

##kerberos Auth should be passed

Auth Failed

Snippet to Reproduce the Problem

client = KazooClient("10.8.8.6:24000,10.8.8.7:24000", sasl_options={"mechanism": "GSSAPI", "service":"zookeeper"})
client.start()

Logs with logging in DEBUG mode

AUTH_FAILED closing: Unknown error: (('Unspecified GSS failure. Minor code may provide more information', 851968), ('Clock skew too great', -1765328347))

Specifications

• Kazoo version: 2.8.0
• Python version: 3.6
• Zookeeper version:
• OS: Centos7

Steps: 1. login kerberos with kinit -kt keytab_file principle 2. check kerberos auth status with klist, its ok 3. run the scripts above

The root cause I found is that: The service send to kerberos is "[email protected]" (pure_sasl-0.6.2-py3.6.egg/puresasl/mechanisms.py):

class GSSAPIMechanism(Mechanism):
    name = 'GSSAPI'
    score = 100
    qops = QOP.all

    allows_anonymous = False
    uses_plaintext = False
    active_safe = True

    def __init__(self, sasl, principal=None, **props):
        Mechanism.__init__(self, sasl)
        self.user = None
        self._have_negotiated_details = False
        self.host = self.sasl.host
        self.service = self.sasl.service
        self.principal = principal
        self._fetch_properties('host', 'service')

        krb_service = '@'.join((self.service, self.host))         ##self.host here should be hostname rather than host ip

Actually, the correct krb_service should be "zookeeper/hostName@Realm" or "[email protected]". for example, [email protected]

KazooClient should take serverFQDN as the input and propagate to puresasl. serverFQDN - the fully qualified domain name of the server (e.g. “serverhost.example.com”).

[tianchao-haohan]

Workaround:

1. Input service as "zookeeper@serverFQDN"
2. Change the code:

class GSSAPIMechanism(Mechanism):
    name = 'GSSAPI'
    score = 100
    qops = QOP.all

    allows_anonymous = False
    uses_plaintext = False
    active_safe = True

    def __init__(self, sasl, principal=None, **props):
        Mechanism.__init__(self, sasl)
        self.user = None
        self._have_negotiated_details = False
        self.host = self.sasl.host
        self.service = self.sasl.service
        self.principal = principal
        self._fetch_properties('host', 'service')

        krb_service = self.service
        if krb_service.find ('@') <= 0 :
            krb_service = '@'.join((self.service, self.host))
    

[ceache]

Clock skew is too great looks like a time synchronization error, not a bad principal issue. Could you double check that this is the actual error you get from the SASL/Kerberos libraries?

Separately, to help us reproduce this issue, could you please detail the exact configuration of your setup?

• Client / sever respective hostnames and IP
• Kerberos principals as returned by klist on both client and server
• Zk server Jaas config, if possible

Cheers,

On Thu, Jan 21, 2021, 22:37 tianchao-haohan [email protected] wrote:

Workaround:

1. Input service as "zookeeper@serverFQDN"
2. Change the code:

class GSSAPIMechanism(Mechanism): name = 'GSSAPI' score = 100 qops = QOP.all

allows_anonymous = False
uses_plaintext = False
active_safe = True

def __init__(self, sasl, principal=None, **props):
    Mechanism.__init__(self, sasl)
    self.user = None
    self._have_negotiated_details = False
    self.host = self.sasl.host
    self.service = self.sasl.service
    self.principal = principal
    self._fetch_properties('host', 'service')

    krb_service = '@'.join((self.service, self.host))         ##self.host here should be hostname rather than host ip

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/python-zk/kazoo/issues/636#issuecomment-765098707, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAIFTHRBGNDVLZI724FP2M3S3DXH3ANCNFSM4WL3RDRA .

[tianchao-haohan]

• Client / sever respective hostnames and IP
  • client hostname: localhost.localdomain,
  • client ip: 10.87.9.15
  • server hostname: hw-ker-node1
  • server ip: 10.8.8.6
• Kerberos principals as returned by klist on both client and server
  • client:

  [c4dev@localhost ~]$ klist
  Ticket cache: KEYRING:persistent:1000:1000
  Default principal: [email protected]
  
  Valid starting       Expires              Service principal
  01/25/2021 14:33:42  01/26/2021 14:33:23  krbtgt/[email protected]
      renew until 01/25/2021 15:03:42
  
  

  • server: empty
• Zk server Jaas config, if possible

Client {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=false
useTicketCache=true
debug=false;
};

[tianchao-haohan]

@ceache hi, is there any update on this issue?

[ceache]

There are a number of issues or at least confusing points of data about your setup. So I am going to go about how things should look like.

First it seems your client's Kerberos environment is set up properly: [email protected]

There are, however, a couple of issues with the information you gave me about the server.

• server hostname: hw-ker-node1

• server ip: 10.8.8.6

• Zk server Jaas config, if possibleClient { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=false useTicketCache=true debug=false; };

So, first, your server(s) should all have a keytab for zookeeper/$ [email protected] . you can change the service from "zookeeper" to whatever you want but you must then use the service="yourservice" parameter in the sasl_options. It could also be in a separate Kerberos realm, but X-realm setup are beyond the scope of this issue.

Once you have the keytab at hand, you need a "Server" section in the Zookeeper server JAAS config. The "Client" section only pertains to the Java Zookeeper client, here you need to configure the Server component. Below is based on what we use in our integration suite:

Server {

com.sun.security.auth.module.Krb5LoginModule required debug=true isInitiator=false useKeyTab=true keyTab="/path/to/the/above/keytab/file" storeKey=true useTicketCache=false principal="zookeeper/[email protected]"; };

Once Zookeeper is setup with this file, you should be able to connect from Kazoo, simply by passing hosts=[hostname], sasl_options={"mechanism"="GSSAPI, service="zookeeper"} to KazooClient.

As for your point about using hostname instead of host IP, this is actually the recommended way to connect to a Kerberized service. Kazoo will pass the hostname if it was provided. That being said, SASL and Kerberos do not really care, as long as your server has that principal in its keytab (e.g. "zookeeper/[email protected] ").

I hope this helps.

-- Charles-Henri de Boysson