Unable to ingest data from Databricks/Hive

[sachinwadhwa]

Describe the bug Unable to ingest data from Databricks using pyhive method specified at https://datahubproject.io/docs/generated/ingestion/sources/hive/

It works fine using python on desktop but UI and docker images both missing databricks-dbapi[hive,sqlalchemy] and acryl-datahub[hive] libraries.

To Reproduce Steps to reproduce the behavior: as above

Expected behavior Include the python libraries databricks-dbapi[hive,sqlalchemy] to ingestion and acryl-datahub-actions docker containers.

[maggiehays]

Hi @anshbansal! Please take a look at this one - let's add in all known ingestion-related dependencies to managed ingestion so we can avoid these types of errors going forward.

[sachinwadhwa]

This issue is still not fixed. Its linked to incorrect fix.

We still get following error while ingesting hive data using UI method (0.8.39):

"NoSuchModuleError: Can't load plugin: sqlalchemy.dialects:databricks.pyhive\n"

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity. If you believe this is still an issue on the latest DataHub release please leave a comment with the version that you tested it with. If this is a question/discussion please head to https://slack.datahubproject.io. For feature requests please use https://feature-requests.datahubproject.io

[github-actions[bot]]

This issue was closed because it has been inactive for 30 days since being marked as stale.