Spark Thrift Server cannot use external metastore

[kk17]

Describe the bug I am running standalone spark version 2.3.2 in docker with ranger-hive-plugin 0.5.3. I use a extend MySQL database for store hive metastore data. I found out that Spark Thrift Server cannot use external metastore for excuting SQL statements after enabling spark configuration: spark.sql.extensions=org.apache.ranger.authorization.spark.authorizer.RangerSparkSQLExtension . If I remove the above configuration Spark Thrift Server can use external metastore but the ranger-hive-plugin is not enabled. This issue does not happen when using spark-sql and spark-shell.

To Reproduce Steps to reproduce the behavior: copied ranger*.xml conf files to spark_home/conf, copied ranger-hive*.jars to spark_home/jars along with spark-authorizer jar. gave full permissions to all xml and jar files. Modified conf xml files as advised. Environments are spark standalone 2.3.2, with ranger-hive-plugin 0.5.3, ranger-admin 0.5.3. Tried running show databases command in beeline.

Expected behavior Spark Thrift Server can use external metastore after enable ranger-hive-plugin.

Additional context Gist for configuration files: Spark-SQL-Ranger

[yaooqinn]

Spark Thrift Server is not guaranteed you may try kyuubi https://github.com/yaooqinn/kyuubi