Use separated libduckdb.so rpm / deb

[Vonng]

trafficstars

How about using a separate libduckdb.so in /usr/lib64/ instead of an embedded one?

libduckdb.so is independent of the PostgreSQL major version. If you plan to install pg_duckdb_16 and pg_duckdb_17 (in the future) simultaneously, it will create two copies of the library, and which are quite large.

Moreover, other DuckDB-related extensions can utilize the same shared library, which could also significantly reduce build time.

Example of RPM packaging spec: https://github.com/pgsty/pgsql-rpm/blob/main/rpmbuild/SPECS/libduckdb.spec

Pre-built binaries for EL system (Debian/Ubuntu can use the official so lib directly):

• https://repo.pigsty.cc/yum/pgsql/el8.x86_64/libduckdb-1.0.0-1PIGSTY.el8.x86_64.rpm
• https://repo.pigsty.cc/yum/pgsql/el9.x86_64/libduckdb-1.0.0-1PIGSTY.el9.x86_64.rpm

[wuputah]

At the moment this isn't in the cards. The plan at the moment for cached_httpfs (see #88) is to provide a version of duckdb with customized extensions that are under development -- though strictly speaking this functionality is optional, you would lose the functionality built there. We've also purposefully picked up some changes in duckdb that are post-1.0.0, so this would be more possible only once 1.1.0 is released.

[Vonng]

Thank you for your response. I am very excited about the potential of the extension to harness the strengths of both PG and DuckDB. I eagerly anticipate the new features in the upcoming version duckdb 1.1 and pg_duckdb 0.1.0

I would also like to inquire if there are any plans to distribute binary packages. I recently packaged version 0.0.1 as an RPM, which can be found here: https://github.com/pgsty/pgsql-rpm/blob/main/rpmbuild/SPECS/pg_duckdb.spec. and binaries:

• https://repo.pigsty.cc/yum/pgsql/el8.x86_64/pg_duckdb_16-0.0.1-1PIGSTY.el8.x86_64.rpm
• https://repo.pigsty.cc/yum/pgsql/el9.x86_64/pg_duckdb_16-0.0.1-1PIGSTY.el9.x86_64.rpm

However, I encountered some issues when compiling on Debian 12 and Ubuntu 22.04. If there's any way I can assist with this or future efforts, I would be delighted to contribute.

Best regards

[wuputah]

Thanks for taking a look into the build process! Right now I can't speak to plans for binaries -- we'll worry about that more closer to release. I'd suggest any binaries right now should have a -dev flag and/or a +commit hash on them since I assume you're building off some specific commit.

Please feel free to report those compilation issues as bugs (or, even better, send PRs to fix them if you can!) as those should be supported platforms.

[wuputah]

Closing in favor of consolidating in #154 and the discussion here otherwise wrapping up, but would be happy to accept your help in building packages!

[Vonng]

Currently, there are three DuckDB-related extensions: duckdb_fdw, pg_mooncake, and pg_duckdb, each using its own version of libduckdb. Consequently, it is not feasible to install these extensions simultaneously. Is there a possibility for pg_duckdb to utilize a standalone libduckdb shared library instead of building its own in the PostgreSQL library directory? Adopting a shared library could significantly reduce compilation times and package sizes, and also eliminate extension conflicts.

I've already facilitated the transition for duckdb_fdw and suggested this approach to the authors of pg_mooncake at thisGitHub issue.

[Bodobolero]

I suggest to use version map approach used here https://github.com/neondatabase/neon/pull/10915