PyHive icon indicating copy to clipboard operation
PyHive copied to clipboard

Dots in column names raises assertion error in sqlachemy hive

Open kotofos opened this issue 4 years ago • 0 comments

Dots in column names get counted as separators between db name, table name and column name.

Reason is code

dot_count = result.count('.')
assert dot_count in (0, 1, 2), "Unexpected visit_column result {}".format(result)

do not take care about quotes. See pyhive.sqlalchemy_hive.HiveCompiler.visit_column

In order to reproduce column should have two dots table.`col.id.part`, or one dot plus explicit dbname prefix dbname.table.`col.id`

from databricks_dbapi.sqlalchemy_dialects.base import DatabricksDialectBase
from sqlalchemy.ext.declarative import declarative_base

from sqlalchemy import Column, Integer
from sqlalchemy.orm import Query

Base = declarative_base()


class Test(Base):
    __tablename__ = 'test'
    id = Column(Integer, name='col.id.part', primary_key=True)


q = Query(Test)
# SELECT test.`col.id.part` FROM test
q.statement.compile(dialect=DatabricksDialectBase())
Traceback (most recent call last):
  File "/home/kotofos/.config/JetBrains/PyCharm2021.2/scratches/scratch_1.py", line 16, in <module>
    q.statement.compile(dialect=DatabricksDialectBase())
  File "<string>", line 1, in <lambda>
  File "/home/kotofos/Documents/slashdb/slashdb_venv/lib64/python3.6/site-packages/sqlalchemy/sql/elements.py", line 481, in compile
    return self._compiler(dialect, bind=bind, **kw)
  File "/home/kotofos/Documents/slashdb/slashdb_venv/lib64/python3.6/site-packages/sqlalchemy/sql/elements.py", line 487, in _compiler
    return dialect.statement_compiler(dialect, self, **kw)
  File "/home/kotofos/Documents/slashdb/slashdb_venv/lib64/python3.6/site-packages/sqlalchemy/sql/compiler.py", line 592, in __init__
    Compiled.__init__(self, dialect, statement, **kwargs)
  File "/home/kotofos/Documents/slashdb/slashdb_venv/lib64/python3.6/site-packages/sqlalchemy/sql/compiler.py", line 322, in __init__
    self.string = self.process(self.statement, **compile_kwargs)
  File "/home/kotofos/Documents/slashdb/slashdb_venv/lib64/python3.6/site-packages/sqlalchemy/sql/compiler.py", line 352, in process
    return obj._compiler_dispatch(self, **kwargs)
  File "/home/kotofos/Documents/slashdb/slashdb_venv/lib64/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 96, in _compiler_dispatch
    return meth(self, **kw)
  File "/home/kotofos/Documents/slashdb/slashdb_venv/lib64/python3.6/site-packages/sqlalchemy/sql/compiler.py", line 2177, in visit_select
    for name, column in select._columns_plus_names
  File "/home/kotofos/Documents/slashdb/slashdb_venv/lib64/python3.6/site-packages/sqlalchemy/sql/compiler.py", line 2177, in <listcomp>
    for name, column in select._columns_plus_names
  File "/home/kotofos/Documents/slashdb/slashdb_venv/lib64/python3.6/site-packages/sqlalchemy/sql/compiler.py", line 1929, in _label_select_column
    return result_expr._compiler_dispatch(self, **column_clause_args)
  File "/home/kotofos/Documents/slashdb/slashdb_venv/lib64/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 96, in _compiler_dispatch
    return meth(self, **kw)
  File "/home/kotofos/Documents/slashdb/slashdb_venv/lib64/python3.6/site-packages/pyhive/sqlalchemy_hive.py", line 161, in visit_column
    assert dot_count in (0, 1, 2), "Unexpected visit_column result {}".format(result)
AssertionError: Unexpected visit_column result `test`.`col.id.part`

kotofos avatar Sep 27 '21 11:09 kotofos