iceberg icon indicating copy to clipboard operation
iceberg copied to clipboard

Make HiveCatalog inheritable for custom IMetaStoreClient

Open JingsongLi opened this issue 4 years ago • 4 comments

Our internal has a custom meta store which implements Hive IMetaStoreClient, I think it is good to reuse Iceberg HiveCatalog:

  • Change HiveMetaStoreClient to IMetaStoreClient. (Actually depends on IMetaStoreClient is enough in Iceberg?)
  • Make HiveClientPool pluggable in HiveCatalog, maybe a new constructor like: public HiveCatalog(String name, HiveClientPool clients, Configuration conf).

JingsongLi avatar Sep 17 '20 03:09 JingsongLi

  • Change HiveMetaStoreClient to IMetaStoreClient. (Actually depends on IMetaStoreClient is enough in Iceberg?)

IMetaStoreClient definitely should be enough for Iceberg. We still have to keep the HMS for testing though (or need a replacement implementation)

pvary avatar Sep 17 '20 07:09 pvary

I also have very similar requirement. I'm not using hadoop or spark I just want to interact with the S3 and HiveMetaStore. Currently there is no way to use HiveMetaStoreClient and use iceberg. Any idea when will this be implemented?

kesavkolla avatar Dec 06 '20 14:12 kesavkolla

@kesavkolla: This should not be that complicated. Currently I am focusing on other parts of the code, but if you want to try out yourself and contribute the code, I would be happy to review your change.

Thanks, Peter

pvary avatar Dec 07 '20 16:12 pvary

This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.

github-actions[bot] avatar Feb 25 '24 00:02 github-actions[bot]

This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale'

github-actions[bot] avatar Mar 11 '24 00:03 github-actions[bot]