using pyiceberg with kerberized hive metastore

[saidixith002]

Question

Hi, Can anyone share examples of using pyiceberg with a kerberized hive metastore?

raise TTransportException(type=TTransportException.END_OF_FILE,
thrift.transport.TTransport.TTransportException: TSocket read 0 bytes

same question posted here - https://github.com/apache/iceberg/issues/6229

Thanks!

[Fokko]

Hey @saidixith002 this is currently not supported, but can be added. I see some Kerberos references in ThriftHiveMetastore.py, such as get_delegation_token and send_get_delegation_token. Are you interested in contributing Kerberos support?

[Ohadkaiser]

Hi! Was there any changes to this issue in the last months?

[wuxiaocheng0506]

Hello, any update? We also need kerberos support.

[awdavidson]

It also looks like the hive catalogs configuration does not support HA HMS.

This configuration uri: thrift://master1.url:9083,thrift://master2.url:9083 throws a ValueError

ValueError: Port could not be cast to integer value as '9083,thrift:'

pyiceberg version is 0.6.1

[awdavidson]

It also looks like the hive catalogs configuration does not support HA HMS.

This configuration uri: thrift://master1.url:9083,thrift://master2.url:9083 throws a ValueError

ValueError: Port could not be cast to integer value as '9083,thrift:'

pyiceberg version is 0.6.1

https://github.com/apache/iceberg-python/pull/752 will address the above

[kevinjqliu]

hey folks, #1634 added supports for kerberize hive metastore. Please check it out and let us know if there's any issues

[dev-sajal]

Hey, just checking if someone was able to use it still? I am unable to access metastore yet, facing the same issue as the initial thread. Also, is there a standardized way of passing in Kerberos principal and keytab into the client?

[dsully]

+1 - the documentation here is severely lacking.

[kevinjqliu]

its difficult to setup and test a kerberize HMS. Would love to get the community's help on this :)

I think https://py.iceberg.apache.org/configuration/#hive-catalog would be a good place to document this