amoro icon indicating copy to clipboard operation
amoro copied to clipboard

[Bug]: For Kyuubi services accessed through the command line, how should users switch between different catalogs? Does this operation have to be done by modifying the Kyuubi configuration file? ​​

Open XingChenBigData opened this issue 6 months ago • 4 comments

What happened?

Multiple catalogs are configured in the Amoro environment, and the Amoro Web Terminal supports users to dynamically switch between these catalogs. For Kyuubi services accessed through the command line, how should users switch between different catalogs? Does this operation have to be done by modifying the Kyuubi configuration file, or can it be done directly by executing Spark-SQL statements? ​​

Affects Versions

0.7.1

What table formats are you seeing the problem on?

No response

What engines are you seeing the problem on?

No response

How to reproduce

No response

Relevant log output


Anything else

No response

Are you willing to submit a PR?

  • [ ] Yes I am willing to submit a PR!

Code of Conduct

  • [x] I agree to follow this project's Code of Conduct

XingChenBigData avatar Jun 08 '25 10:06 XingChenBigData

#vi /etc/kyuubi/conf/kyuubi-defaults.conf spark.sql.defaultCatalog=mixed_catalog ##First catalog spark.sql.catalog.mixed_catalog=org.apache.amoro.spark.MixedFormatSparkCatalog spark.sql.catalog.mixed_catalog.url=thrift://172.18.0.4:1260/mixed_catalog

##Second catalog spark.sql.catalog.kyuubi_mixed_iceberg_catalog=org.apache.amoro.spark.MixedFormatSparkCatalog spark.sql.catalog.kyuubi_mixed_iceberg_catalog.url=thrift://172.18.0.4:1260/kyuubi_mixed_iceberg_catalog

#Restart kyuubi /opt/kyuubi/bin/kyuubi restart

#Enter the kyuubi command line kyuubi-beeline -u "jdbc:hive2://127.0.0.1:10009/"

0: jdbc:hive2://127.0.0.1:10009/> show catalogs; +----------------+ | catalog | +----------------+ | mixed_catalog | | tpcds | | tpch | +----------------+

Why is there no separate kyuubi_mixed_iceberg_catalog catalog?

XingChenBigData avatar Jun 08 '25 10:06 XingChenBigData

#应用 Spark Extensions spark.sql.extensions=org.apache.amoro.spark.MixedFormatSparkExtensions #配置 Amoro Catalog spark.sql.defaultCatalog=mixed_catalog
spark.sql.catalog.mixed_catalog=org.apache.amoro.spark.MixedFormatSparkCatalog spark.sql.catalog.mixed_catalog.url=thrift://172.18.0.4:1260/mixed_catalog

#第二个catalog
spark.sql.catalog.kyuubi_mixed_iceberg_catalog=org.apache.amoro.spark.MixedFormatSparkCatalog spark.sql.catalog.kyuubi_mixed_iceberg_catalog.url=thrift://172.18.0.4:1260/kyuubi_mixed_iceberg_catalog

XingChenBigData avatar Jun 09 '25 02:06 XingChenBigData

#编辑kyuubi配置文件,增加下面内容 #vi /etc/kyuubi/conf/kyuubi-defaults.conf spark.sql.defaultCatalog=amoro_external_catalog_for_iceberg spark.sql.extensions=org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions spark.sql.catalog.spark_catalog=org.apache.iceberg.spark.SparkSessionCatalog spark.sql.catalog.spark_catalog.type=hive spark.sql.catalog.amoro_external_catalog_for_iceberg=org.apache.iceberg.spark.SparkCatalog spark.sql.catalog.local.type=hive spark.sql.catalog.local.uri=thrift://hadoop-master1.orb.local:9083 spark.sql.catalog.amoro_external_catalog_for_iceberg.warehouse=hdfs://hadoop-master1.orb.local:8020/warehouse

XingChenBigData avatar Jun 09 '25 02:06 XingChenBigData