dbt-duckdb icon indicating copy to clipboard operation
dbt-duckdb copied to clipboard

possibility to use postgres_scan isntead of attach

Open dberardo-com opened this issue 6 months ago • 2 comments

postgres_scan was way faster than attach and also gave to possibility to attach just a single table ... is this still possible to use ?

dberardo-com avatar Jun 17 '25 16:06 dberardo-com

yes, absolutely; if you load the postgres extension, the postgres_scan function is available and you can use it as you regularly would to query a single table from a dbt model; if you want to abstract the table via a dbt source, you can do that using the same external_location config option that we use for doing things like reading in tables via the read_csv or read_parquet functions, documented here: https://github.com/duckdb/dbt-duckdb?tab=readme-ov-file#reading-from-external-files

jwills avatar Jun 17 '25 16:06 jwills

ok i will give all this a try, at the moment i was defining the "attach" option within the profiles.yaml file but i believe i was doing it wrongly because there i can only define outputs and not sources/inputs ... am i right ?

so since i need to source data from those table i might be using a source with the postgres_scan command. am i right ? do you have any example on how to use the postgres_scan in a source definition ?

cheers

dberardo-com avatar Jun 19 '25 13:06 dberardo-com