[Bug]: Long column names are truncated
What happened?
Observed in https://github.com/ibis-project/ibis/issues/8393.
Long column names appear to be truncated when using duckdb_engine to fetch results.
Reproducible example:
import sqlalchemy as sa
long_name = "a_very_very_very_very_very_very_very_very_very_long_column_name"
eng = sa.create_engine("duckdb:///test.db")
with eng.begin() as con:
con.exec_driver_sql(f"CREATE OR REPLACE TABLE t ({long_name} INT)")
t = sa.Table("t", sa.MetaData(), autoload_with=eng)
t1 = sa.select(t.c[0]).select_from(t).subquery()
query = sa.select(t1.c[long_name])
with eng.begin() as con:
result = con.execute(query)
batch = result.connection.connection.fetch_record_batch()
assert batch.schema.names[0] == long_name, batch.schema.names[0]
DuckDB Engine Version
0.10.0
DuckDB Version
0.9.2, 0.10.0
SQLAlchemy Version
1.4.51
Relevant log output
No response
Code of Conduct
- [X] I agree to follow this project's Code of Conduct
Looks like this is a default inherited from Postgres: https://github.com/sqlalchemy/sqlalchemy/blob/1c58fe53b6fd069cbb82955ddaf9eb5405076146/lib/sqlalchemy/dialects/postgresql/base.py#L2941
I wonder if this is another nail in the coffin for subclassing from that dialect.
In the next Ibis release we're moving away from SQLAlchemy entirely to avoid the extra layer and myriad dependencies (which Ibis already has plenty of!).
duckdb_engine has been great to work with!
Given that it's in the "base" Postgres dialect, rather than the psycopg2 one, I'm not sure it is
I saw in the linked issue, congrats!