dbt-duckdb icon indicating copy to clipboard operation
dbt-duckdb copied to clipboard

DuckDB relation failing as Python model return type

Open Soloniss opened this issue 1 year ago • 1 comments

When returning a DuckDB relation as a return type for my python model I get an error stating that the dataframe cannot be found

Catalog Error: Table with name df does not exist! Did you mean "pg_depend"? LINE 1: ... my_transform__dbt_tmp" as select * from df 

However, after converting the relation to a pandas dataframe or arrow table, things work as expected. My assumption is there's something wrong with the DuckDB connection which closes the relation, or the relation being out of scope. Any tips here?

Soloniss avatar Oct 31 '24 22:10 Soloniss

Hey @Soloniss, apologies for the lag here-- I tried to play with this a bit and I couldn't reproduce it locally, but then I promptly got busy with some non-DuckDB work stuff and forgot to post about it. Is there any easy way to repro this that you've come across? I have a couple of functional tests that exercise variations of the scenario you described and they all work AFAICT, but if I've missed something I would very much like to fix it: https://github.com/duckdb/dbt-duckdb/blob/master/tests/functional/adapter/test_python_model.py#L116

jwills avatar Nov 11 '24 23:11 jwills