pyspark.sql.utils.AnalysisException: "Table or view not found: when creating an expectations suite

[jpoullet2000]

Describe the bug Hive metastore is not found, and so the hive table is not found.

To Reproduce When running

great_expectations suite scaffold ge_demo_tbl.warning

I always get pyspark.sql.utils.AnalysisException: "Table or view not found:

Expected behavior It seems that it cannot connect to the hive metastore.

I tried to set up the environment variables HADOOP_CONF_DIR, SPARK_HOME and HIVE_CONF_DIR (using export in bash, or even using os.environ[VAR] = path) in vain.

I also tried to replace the line builder = SparkSession.builder in core/util.py by builder = SparkSession.builder.enableHiveSupport() without more success.

Environment (please complete the following information):

• Operating System: Linux
• Great Expectations Version: 0.13.37

Additional context My great_expectations.yml file

datasources:
  spark_dataframe:
    data_asset_type:
      class_name: SparkDFDataset
      module_name: great_expectations.dataset
    batch_kwargs_generators:
      spark_sql_query:
        class_name: QueryBatchKwargsGenerator
        queries:
          'explanation_table': 'select count(*) from mydb.mytable' 
    module_name: great_expectations.datasource
    class_name: SparkDFDatasource

[talagluck]

Thanks for raising this issue, @jpoullet2000! We are not currently able to prioritize V2 issues, and additionally, we are not able to prioritize Hive issues. That said, we always welcome community contributions! Please feel free to reach out if you would like to contribute.

[github-actions[bot]]

Is this issue still relevant? If so, what is blocking it? Is there anything you can do to help move it forward?

This issue has been automatically marked as stale because it has not had recent activity.

It will be closed if no further activity occurs. Thank you for your contributions 🙇

[talagluck]

Thanks again for raising! Closing, since this appears to be linked to V2, as mentioned above.