postgres_scanner icon indicating copy to clipboard operation
postgres_scanner copied to clipboard

Read native parquet files from Postgres using DuckDB

Open sushrut141 opened this issue 7 months ago • 0 comments

I'm the author of the pg_analytica extension that enables periodic parquet exports for Postgres tables. See: https://github.com/sushrut141/pg_analytica

The extension currently relies on the parquet_fdw extension to query the parquet files. If DuckDB implements it's own Foreign Data Wrapper for Postgres then querying would be greatly sped up compared to parquet_fdw since DuckDB optimiser is known for better pipelining / vectorisation of queries.

I was wondering if this is something we could collaborate on and package the two extensions as a complete analytics package for Postgres instances. I would be happy to contribute to the creation of this FDW if you can share some pointers.

Thanks

sushrut141 avatar Jul 06 '24 10:07 sushrut141