duckdb_engine icon indicating copy to clipboard operation
duckdb_engine copied to clipboard

Question: Any way to create engine from existing connection?

Open NickCrews opened this issue 1 year ago • 2 comments

What happened?

See https://github.com/duckdb/duckdb/discussions/14549#discussion-7364961. Any way to go from duckdb.DuckDBPyConnection to a sqlalchemy.engine? I have to make my own little wrapper class that provides the needed methods for sqlalchemy? Thanks!

DuckDB Engine Version

NA

DuckDB Version

NA

SQLAlchemy Version

No response

Relevant log output

No response

Code of Conduct

  • [X] I agree to follow this project's Code of Conduct

NickCrews avatar Oct 25 '24 18:10 NickCrews

I can't say I'd really recommend this, but you can do this to work around that error:

from duckdb_engine import ConnectionWrapper

engine = sa.create_engine("duckdb://", creator=lambda: ConnectionWrapper(con))

this class just patches the duckdb connection to be what sqlalchemy expects from postgres/psycopg2

Mause avatar Oct 27 '24 14:10 Mause

Thanks!

I can't say I'd really recommend this,

because you'd like to keep ConnectionWrapper as non-public, and you might make breaking changes at any time? That seems like a good idea.

What would you think about if you encapsulated that into a more simple API, and then exposed that publicly:

def create_engine(con: duckdb.DuckDBPyConnection) -> sqlalchemy.engine.Engine:
    return sa.create_engine("duckdb://", creator=lambda: ConnectionWrapper(con))

I'm not that familiar with sqlalchemy's API though, are users then gonna want to pass in various kwargs into creat_engine?

NickCrews avatar Oct 28 '24 19:10 NickCrews