Yi-Kuan Lee comments

Results 7 comments of


                                            Yi-Kuan Lee

Create iceberg table throw exception when Hive Ranger is enabled

By looking into Ranger access log. Hive SQL policy for DB and Table been triggered and allowed. So something might be related with HiveIcebergStorageHandler the URI for Auth for RWSTORAGE

Create iceberg table throw exception when Hive Ranger is enabled

``` 2022-09-14T13:15:15,949 WARN com.ibm.biginsights.catalog.translator.hive.HiveExceptionTranslator [Thread-21] : [BSL-0-33c23b26d]: Exception: FAILED: Execution Error, return code 40000 from org.apache.hadoop.hive.ql.ddl.DDLTask. org.apache.iceberg.exceptions.CommitStateUnknownException: MetaException(message:Permission denied: user [db2iuser] does not have [RWSTORAGE] privilege on [null://null]) Cannot determine...

Create iceberg table throw exception when Hive Ranger is enabled

Not sure it related, the table created successfully had InputFormat and OutputFormat properties ``` 0: jdbc:hive2://cms.dv.net:2181/default> describe formatted t11_iceberg; INFO : Compiling command(queryId=hive_20220922133753_360c62be-8f68-455a-a27c-e553988ab90f): describe formatted t11_iceberg DEBUG : Encoding valid...

Create iceberg table throw exception when Hive Ranger is enabled

Also, we can see Hive metastore log show iceberg table report AllRequestedHiveResources={db2iuser/t32_iceberg; null://null; } not sure why iceberg won't return full hdfs directory but null://null instead. ``` 2022-09-22 17:30:48,009 DEBUG...

Create iceberg table throw exception when Hive Ranger is enabled

Finally, i think i featured out the root caused from metastore log it seems like same as https://issues.apache.org/jira/browse/HIVE-25964| which iceberg 0.14 did not implement getURIForAuth() method. ``` 2022-09-22 17:30:47,983 ERROR...

Create iceberg table throw exception when Hive Ranger is enabled

i am going to implement getURIForAuth() method to HiveIcebergStorageHandler.java and test on Ranger. If it works ( can create an iceberg table with Ranger policy ) . i am going...

Create iceberg table throw exception when Hive Ranger is enabled

@nastra , I think i figured out why, IBM patched Cloudera hive 3.1.3 version with newer ql version which eventually included ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveStorageAuthorizationHandler.java which came from https://github.com/apache/hive/pull/1960/files#diff-8527b42317c96d8d571df7271740fd2d233ebd14d9b2d71f547525ca2bd3ff9b Therefore When Hive alter/create...