'trino' not in table.dialect_options

[justineyster]

A clear and concise description of what the bug is.

This seems related to #20043.

Our Superset/Trino integration has been broken by a recent update but I can't yet figure out a fix.

Our dashboards and graphs that utilize Trino are failing with DB engine Error: 'TableClause' object has no attribute 'dialect_options'.

In the logs I'm also seeing 'trino' not in table.dialect_options and WARNING:superset.db_engine_specs:Unable to load SQLAlchemy dialect <class 'es.opendistro.sqlalchemy.ESHTTPSDialect'>: No module named 'sqlalchemy_trino'.

We're on version 1.5.1 of Superset installed on a Python 3.9 Docker image via pip with extras, i.e. pip install apache-superset[db2,druid,elasticsearch,mysql,postgres,thumbnails,trino].

I see there has been a lot of activity lately around the Trino engine in this repo. I'm hoping someone with more intimate understanding of the SQLAlchemy dialect situation might be able to shed some light on what's going on, and how to fix it.

How to reproduce the bug

1. Create a Superset dashboard with a Trino data source
2. Attempt to load the dashboard

Expected results

Dashboard loads and can access Trino.

Actual results

Errors mentioned above (stack trace at bottom).

Screenshots

Environment

(please complete the following information):

• browser type and version: Chrome
• superset version: 1.5.1
• python version: 3.9
• node.js version: N/A
• any feature flags active:

Checklist

Make sure to follow these steps before submitting your issue - thank you!

• [x] I have checked the superset logs for python stacktraces and included it here as text if there are any.
• [ ] I have reproduced the issue with at least the latest released version of superset.
• [x] I have checked the issue tracker for the same issue and I haven't found one similar.

Additional context

AttributeError: 'TableClause' object has no attribute 'dialect_options'
2022-09-21 15:28:03,660:ERROR:superset.views.base:'TableClause' object has no attribute 'dialect_options'
Traceback (most recent call last):
File "/opt/app-root/lib64/python3.9/site-packages/flask/app.py", line 1950, in full_dispatch_request
  rv = self.dispatch_request()
File "/opt/app-root/lib64/python3.9/site-packages/flask/app.py", line 1936, in dispatch_request
  return self.view_functions[rule.endpoint](**req.view_args)
File "/opt/app-root/lib64/python3.9/site-packages/flask_appbuilder/security/decorators.py", line 104, in wraps
  return f(self, *args, **kwargs)
File "/opt/app-root/lib64/python3.9/site-packages/superset/views/base_api.py", line 112, in wraps
  raise ex
File "/opt/app-root/lib64/python3.9/site-packages/superset/views/base_api.py", line 109, in wraps
  duration, response = time_function(f, self, *args, **kwargs)
File "/opt/app-root/lib64/python3.9/site-packages/superset/utils/core.py", line 1468, in time_function
  response = func(*args, **kwargs)
File "/opt/app-root/lib64/python3.9/site-packages/superset/utils/log.py", line 245, in wrapper
  value = f(*args, **kwargs)
File "/opt/app-root/lib64/python3.9/site-packages/superset/charts/data/api.py", line 244, in data
  return self._get_data_response(
File "/opt/app-root/lib64/python3.9/site-packages/superset/charts/data/api.py", line 390, in _get_data_response
  result = command.run(force_cached=force_cached)
File "/opt/app-root/lib64/python3.9/site-packages/superset/charts/data/commands/get_data_command.py", line 43, in run
  payload = self._query_context.get_payload(
File "/opt/app-root/lib64/python3.9/site-packages/superset/common/query_context.py", line 96, in get_payload
  return self._processor.get_payload(cache_query_context, force_cached)
File "/opt/app-root/lib64/python3.9/site-packages/superset/common/query_context_processor.py", line 376, in get_payload
  query_results = [
File "/opt/app-root/lib64/python3.9/site-packages/superset/common/query_context_processor.py", line 377, in <listcomp>
  get_query_results(
File "/opt/app-root/lib64/python3.9/site-packages/superset/common/query_actions.py", line 200, in get_query_results
  return result_func(query_context, query_obj, force_cached)
File "/opt/app-root/lib64/python3.9/site-packages/superset/common/query_actions.py", line 104, in _get_full
  payload = query_context.get_df_payload(query_obj, force_cached=force_cached)
File "/opt/app-root/lib64/python3.9/site-packages/superset/common/query_context.py", line 115, in get_df_payload
  return self._processor.get_df_payload(query_obj, force_cached)
File "/opt/app-root/lib64/python3.9/site-packages/superset/common/query_context_processor.py", line 126, in get_df_payload
  query_result = self.get_query_result(query_obj)
File "/opt/app-root/lib64/python3.9/site-packages/superset/common/query_context_processor.py", line 186, in get_query_result
  result = query_context.datasource.query(query_object.to_dict())
File "/opt/app-root/lib64/python3.9/site-packages/superset/connectors/sqla/models.py", line 1693, in query
  query_str_ext = self.get_query_str_extended(query_obj)
File "/opt/app-root/lib64/python3.9/site-packages/superset/connectors/sqla/models.py", line 835, in get_query_str_extended
  sql = self.database.compile_sqla_query(sqlaq.sqla_query)
File "/opt/app-root/lib64/python3.9/site-packages/superset/models/core.py", line 466, in compile_sqla_query
  sql = str(qry.compile(engine, compile_kwargs={"literal_binds": True}))
File "<string>", line 1, in <lambda>
File "/opt/app-root/lib64/python3.9/site-packages/sqlalchemy/sql/elements.py", line 481, in compile
  return self._compiler(dialect, bind=bind, **kw)
File "/opt/app-root/lib64/python3.9/site-packages/sqlalchemy/sql/elements.py", line 487, in _compiler
  return dialect.statement_compiler(dialect, self, **kw)
File "/opt/app-root/lib64/python3.9/site-packages/sqlalchemy/sql/compiler.py", line 592, in __init__
  Compiled.__init__(self, dialect, statement, **kwargs)
File "/opt/app-root/lib64/python3.9/site-packages/sqlalchemy/sql/compiler.py", line 322, in __init__
  self.string = self.process(self.statement, **compile_kwargs)
File "/opt/app-root/lib64/python3.9/site-packages/sqlalchemy/sql/compiler.py", line 352, in process
  return obj._compiler_dispatch(self, **kwargs)
File "/opt/app-root/lib64/python3.9/site-packages/sqlalchemy/sql/visitors.py", line 96, in _compiler_dispatch
  return meth(self, **kw)
File "/opt/app-root/lib64/python3.9/site-packages/sqlalchemy/sql/compiler.py", line 2201, in visit_select
  text = self._compose_select_body(
File "/opt/app-root/lib64/python3.9/site-packages/sqlalchemy/sql/compiler.py", line 2292, in _compose_select_body
  [
File "/opt/app-root/lib64/python3.9/site-packages/sqlalchemy/sql/compiler.py", line 2293, in <listcomp>
  f._compiler_dispatch(self, asfrom=True, **kwargs)
File "/opt/app-root/lib64/python3.9/site-packages/sqlalchemy/sql/visitors.py", line 96, in _compiler_dispatch
  return meth(self, **kw)
File "/opt/app-root/lib64/python3.9/site-packages/trino/sqlalchemy/compiler.py", line 121, in visit_table
  return self.add_catalog(sql, table)
File "/opt/app-root/lib64/python3.9/site-packages/trino/sqlalchemy/compiler.py", line 132, in add_catalog
  'trino' not in table.dialect_options
AttributeError: 'TableClause' object has no attribute 'dialect_options'

[justineyster]

Note that we only encountered this error after upgrading the version of our UBI Python 3.9 image from 1-57 to 1-73: https://catalog.redhat.com/software/containers/ubi8/python-39/6065b24eb92fbda3a4c65d8f

Reverting back to 1-57 fixes the issue. But we'd much rather be on the latest patched version of the image.

[rylativity]

Exact same issue, error, and stacktrace as above, but using apache/superset:latest image from Docker Hub for testing and development. And also like Justin, I was using Superset with Trino for a while without issue, and this error only started appearing recently.

[iercan]

We have same issue. We are using 1.5.0 docker image. Interestingly it gives this error with certain datasets.

[iercan]

Seem like issue is related with sqlalchemy-trino. I've downgraded it to 0.4.1 from 0.5.0 and problem solved for me.

[justineyster]

@iercan My understanding is that the sqlalchemy-trino project is deprecated and was removed from Superset for that reason. You can see a note about that here, which also states that the project has been merged into the upstream Trino package itself: https://github.com/dungdm93/sqlalchemy-trino. Personally, I'd rather not re-introduce a deprecated dependency.

[iercan]

You can install trino==0.315.0. Seems like latest version is problematic.

[justineyster]

@iercan Thanks for that info! I see that a duplicate issue has been raised here: trinodb/trino-python-client#237

[EMCP]

just a heads up.. I just tried

Superset v2.0.0 Trino 399 HIVE Metastore + S3 parquet files

and the latest trino==0.317.0 library.. it fails just to connect.. so I need to back down to v0.315.0 to connect and attempt to get going at all

[georgewfisher]

Using the latest release of Superset 2.0.0 with Trino 381 I can now load charts without this error. Previously I had this issue.

I have explicitly tried Superset 2.0.0, Trino 381, and "trino": ["trino>=0.317.0"], and that combination works. Trino Python library fix for this issue is here: https://github.com/trinodb/trino-python-client/pull/240

[EMCP]

I was on 398 Trino, now 399.. with that pypi version of the Trino Library I was not even able to connect to Trino in the GUI.. if I can I will try again with 317 driver but for now backed off the 315... I get connected but have now hit some sort of permissions issue possibly .. posted that to Trino slack

On Mon, Oct 10, 2022, 20:22 georgewfisher @.***> wrote:

Using the latest release of Superset 2.0.0 with Trino 381 I can now load charts without this error. Previously I had this issue.

I have explicitly tried Superset 2.0.0, Trino 381, and "trino": ["trino>=0.317.0"], and that combination works. Trino Python library fix for this issue is here: trinodb/trino-python-client#240 https://github.com/trinodb/trino-python-client/pull/240

— Reply to this email directly, view it on GitHub https://github.com/apache/superset/issues/21541#issuecomment-1273668345, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA4FJSQIYGAII344R62VVSLWCRNEPANCNFSM6AAAAAAQSHNJ7M . You are receiving this because you commented.Message ID: @.***>

[iercan]

Latest python client solved this issue. Trino 399 with python client 0.318 works without error now. FYI.

[EMCP]

I can confirm with trino 399 + hive-metastore 3.x + S3 + trino==0.318.0 I can successfully start connecting and building datasets and charts!

[rusackas]

Are people still seeing this in Superset 3.0/3.1? This thread has been quiet for an awfully long time.

[rusackas]

Ok, hoping this is all resolved then. Please provide updated context if this needs to be reopened, or just open a fresh Issue :)