superset icon indicating copy to clipboard operation
superset copied to clipboard
verified publisher icon apache

'str' object does not support item assignment, while Databrciks database connection

Open nagarajmmu opened this issue 3 years ago • 1 comments

Hi Team

I have installed superset with docker container, logged in to superset. I wanted to connect to Azure Databricks. I have followed the below steps.

  1. I have created "requirements-local.txt" file in /superset/docker/ folder.
  2. Added "databricks-dbapi[hive,odbc,sqlalchemy]" in "requirements-local.txt" file.
  3. Ran "docker-compose build --force-rm" command to build.
  4. Ram "docker-compose -f docker-compose-non-dev.yml up" command to run the docker.
  5. Superset is up and running and able to login.

How to reproduce the bug

  1. Go to Data -> "Connect a database"
  2. Select "Databricks SQL Endpoint" in "SUPPORTED DATABASES" dropdown .
  3. Provide "databricks+pyhive://token:{access_token}@{server_hostname}:{port}/{database_name}" URL details from Databricks, for "SQLALCHEMY URI"
  4. Provide "{"connect_args": {"http_path": "sql/protocolv1/o/****"}}" with Http Path, in ADVANCED > Other > ENGINE PARAMETERS.
  5. Click on "TEST CONNECTION"
  6. Below ERROR occurred ERROR: superset_app | SupersetErrorsException superset_app | Traceback (most recent call last): superset_app | File "/app/superset/databases/commands/test_connection.py", line 102, in run superset_app | with database.get_sqla_engine_with_context() as engine: superset_app | File "/usr/local/lib/python3.8/contextlib.py", line 113, in enter superset_app | return next(self.gen) superset_app | File "/app/superset/models/core.py", line 372, in get_sqla_engine_with_context superset_app | yield self._get_sqla_engine(schema=schema, nullpool=nullpool, source=source) superset_app | File "/app/superset/models/core.py", line 396, in _get_sqla_engine superset_app | params["poolclass"] = NullPool superset_app | TypeError: 'str' object does not support item assignment superset_app | superset_app | The above exception was the direct cause of the following exception: superset_app | superset_app | Traceback (most recent call last): superset_app | File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1516, in full_dispatch_request superset_app | rv = self.dispatch_request() superset_app | File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1502, in dispatch_request superset_app | return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) superset_app | File "/usr/local/lib/python3.8/site-packages/flask_appbuilder/security/decorators.py", line 89, in wraps superset_app | return f(self, *args, **kwargs) superset_app | File "/app/superset/views/base_api.py", line 114, in wraps superset_app | raise ex superset_app | File "/app/superset/views/base_api.py", line 111, in wraps superset_app | duration, response = time_function(f, self, *args, **kwargs) superset_app | File "/app/superset/utils/core.py", line 1604, in time_function superset_app | response = func(*args, **kwargs) superset_app | File "/app/superset/utils/log.py", line 265, in wrapper superset_app | value = f(*args, **kwargs) superset_app | File "/app/superset/views/base_api.py", line 84, in wraps superset_app | return f(self, *args, **kwargs) superset_app | File "/app/superset/databases/api.py", line 726, in test_connection superset_app | TestConnectionDatabaseCommand(item).run() superset_app | File "/app/superset/databases/commands/test_connection.py", line 176, in run superset_app | raise DatabaseTestConnectionUnexpectedError(errors) from ex superset_app | superset.databases.commands.exceptions.DatabaseTestConnectionUnexpectedError: [SupersetError(message="'str' object does not support item assignment", error_type=<SupersetErrorType.GENERIC_DB_ENGINE_ERROR: 'GENERIC_DB_ENGINE_ERROR'>, level=<ErrorLevel.ERROR: 'error'>, extra={'engine_name': 'Databricks Interactive Cluster', 'issue_codes': [{'code': 1002, 'message': 'Issue 1002 - The database returned an unexpected error.'}]})] superset_app | 2022-11-29 23:29:08,364:WARNING:superset.views.base:SupersetErrorsException superset_app | Traceback (most recent call last): superset_app | File "/app/superset/databases/commands/test_connection.py", line 102, in run superset_app | with database.get_sqla_engine_with_context() as engine: superset_app | File "/usr/local/lib/python3.8/contextlib.py", line 113, in enter superset_app | return next(self.gen) superset_app | File "/app/superset/models/core.py", line 372, in get_sqla_engine_with_context superset_app | yield self._get_sqla_engine(schema=schema, nullpool=nullpool, source=source) superset_app | File "/app/superset/models/core.py", line 396, in _get_sqla_engine superset_app | params["poolclass"] = NullPool superset_app | TypeError: 'str' object does not support item assignment superset_app | superset_app | The above exception was the direct cause of the following exception: superset_app | superset_app | Traceback (most recent call last): superset_app | File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1516, in full_dispatch_request superset_app | rv = self.dispatch_request() superset_app | File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1502, in dispatch_request superset_app | return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) superset_app | File "/usr/local/lib/python3.8/site-packages/flask_appbuilder/security/decorators.py", line 89, in wraps superset_app | return f(self, *args, **kwargs) superset_app | File "/app/superset/views/base_api.py", line 114, in wraps superset_app | raise ex superset_app | File "/app/superset/views/base_api.py", line 111, in wraps superset_app | duration, response = time_function(f, self, *args, **kwargs) superset_app | File "/app/superset/utils/core.py", line 1604, in time_function superset_app | response = func(*args, **kwargs) superset_app | File "/app/superset/utils/log.py", line 265, in wrapper superset_app | value = f(*args, **kwargs) superset_app | File "/app/superset/views/base_api.py", line 84, in wraps superset_app | return f(self, *args, **kwargs) superset_app | File "/app/superset/databases/api.py", line 726, in test_connection superset_app | TestConnectionDatabaseCommand(item).run() superset_app | File "/app/superset/databases/commands/test_connection.py", line 176, in run superset_app | raise DatabaseTestConnectionUnexpectedError(errors) from ex superset_app | superset.databases.commands.exceptions.DatabaseTestConnectionUnexpectedError: [SupersetError(message="'str' object does not support item assignment", error_type=<SupersetErrorType.GENERIC_DB_ENGINE_ERROR: 'GENERIC_DB_ENGINE_ERROR'>, level=<ErrorLevel.ERROR: 'error'>, extra={'engine_name': 'Databricks Interactive Cluster', 'issue_codes': [{'code': 1002, 'message': 'Issue 1002 - The database returned an unexpected error.'}]})] superset_app | 10.48.4.5 - - [29/Nov/2022:23:29:08 +0000] "POST /api/v1/database/test_connection/ HTTP/1.1" 422 296 "http://[Host_IP]:8088/superset/welcome/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" superset_app | 10.48.4.5 - - [29/Nov/2022:23:29:39 +0000] "GET /api/v1/me/ HTTP/1.1" 200 150 "http://[Host_IP]:8088/superset/welcome/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36"

Please let me know, if I am missing anything while connecting to Azure Databricks.

Please help me to connect to Azure Databricks.

Thanks Nagaraj M M

nagarajmmu avatar Nov 29 '22 23:11 nagarajmmu

Pinging @betodealmeida as someone who knows the intricacies of Databricks x Superset. Please feel free to redirect/shepherd this issue as you see fit :)

rusackas avatar Nov 30 '22 17:11 rusackas

Hi Evan

Thank you for adding betodealmeida

nagarajmmu avatar Nov 30 '22 22:11 nagarajmmu

Same error message trying to connect a hive database on preset.io

"test connection" reports, "Connection looks good!", but "connect" triggers the error.

{
  "errors": [
    {
      "message": "'str' object does not support item assignment",
      "error_type": "GENERIC_DB_ENGINE_ERROR",
      "level": "error",
      "extra": {
        "engine_name": "Apache Hive",
        "issue_codes": [
          {
            "code": 1002,
            "message": "Issue 1002 - The database returned an unexpected error."
          }
        ]
      }
    }
  ]

fullergalway avatar Dec 06 '22 12:12 fullergalway

"test connection" reports, "Connection looks good!", but "connect" triggers the error.

The problem occurs in preset.io for me, but in my superset development environment there is no issue. (Reporting in preset)

fullergalway avatar Dec 06 '22 17:12 fullergalway

Hi fullergalway and Elizabeth Thompson

Thanks for the update, I am waiting for the fix.

Thanks Nagaraj M M

nagarajmmu avatar Dec 08 '22 00:12 nagarajmmu

Hi @eschutho I have tried your code by downloading locally and tested still the issue is not fixed. Please look into it again.

vignesh-maersk2022 avatar Dec 10 '22 21:12 vignesh-maersk2022

I'm assuming this was fixed by now but never closed. Can anyone confirm on Preset and/or Superset 3.x?

rusackas avatar Feb 15 '24 02:02 rusackas

It's been four months (well, almost) and we're on Superset 4.x. I'm still hopeful it's fixed, but certain that the issue is stale. Holler if this needs a revisit or to be reopened for some reason.

rusackas avatar Jun 11 '24 21:06 rusackas