postgres_scanner
postgres_scanner copied to clipboard
Read native parquet files from Postgres using DuckDB
I'm the author of the pg_analytica extension that enables periodic parquet exports for Postgres tables. See: https://github.com/sushrut141/pg_analytica
The extension currently relies on the parquet_fdw extension to query the parquet files. If DuckDB implements it's own Foreign Data Wrapper for Postgres then querying would be greatly sped up compared to parquet_fdw since DuckDB optimiser is known for better pipelining / vectorisation of queries.
I was wondering if this is something we could collaborate on and package the two extensions as a complete analytics package for Postgres instances. I would be happy to contribute to the creation of this FDW if you can share some pointers.
Thanks