extension-template icon indicating copy to clipboard operation
extension-template copied to clipboard

question about building template extension without makefile

Open WMF1997 opened this issue 4 months ago • 0 comments

I manage to build this example quack extension without using default Makefile or CMakeLists.txt. The building procedure is listed in the makrdown file[build_procedure.md](https://github.com/user-attachments/files/17269902/build_procedure.md), and I also list it as below.

I just do not know how to use add valid metadata at the end of the built quack.duckdb_extension. (I use append_metadata.cmake to add it by executing cmake quack.duckdb_extension -DDUCKDB_VERSION=v1.1.0 -DEXTENSION_VERSION=v1.1.0 -P scripts/append_metadata.cmake), but it does not work after executing it and copy the quack.duckdb_extension to ~/.duckdb/extension/v1.1.0/linux_amd64_gcc4/ extension folder, with some extensions in this folder installed before. I use duckdb -unsigned to enter duckdb cli, and then I execute load quack, but it also does not work. the error info implies that the metadata located at the end of the quack.duckdb_extension file is invalid.

The following commands are found by reading cmake-generated files, such as copile_commands.json, extension/quack/CMakeFiles/... inside the generated build/release directory. I managed to execute them step by step, and they worked. (Just do not know how to add valid metadata). If valid metadata is added, we can manage to build the duckdb extension without building duckdb in the same time, if you have built a full duckdb env (for example, you have built a libduckdb_static.a)

Here is my build environment: OS: Ubuntu 22.04 duckdb: v1.1.0 (I directly download the source .zip from github, and then all features related to .git are missing) Toolchain: g++ 11.4.0 + cmake 3.22.1 + make 4.3 (I add _GLIBCXX_USE_CXX11_ABI=0 to build extension, since the duckdb executable (also v1.1.0) downloaded from github is linux_amd64_gcc4)

build extension

/usr/bin/c++ -DDUCKDB_BUILD_LIBRARY -I/home/wmf1997/build_software/extension-template-main/duckdb/src/include -I/home/wmf1997/build_software/extension-template-main/duckdb/third_party/fsst -I/home/wmf1997/build_software/extension-template-main/duckdb/third_party/fmt/include -I/home/wmf1997/build_software/extension-template-main/duckdb/third_party/hyperloglog -I/home/wmf1997/build_software/extension-template-main/duckdb/third_party/fastpforlib -I/home/wmf1997/build_software/extension-template-main/duckdb/third_party/skiplist -I/home/wmf1997/build_software/extension-template-main/duckdb/third_party/fast_float -I/home/wmf1997/build_software/extension-template-main/duckdb/third_party/re2 -I/home/wmf1997/build_software/extension-template-main/duckdb/third_party/miniz -I/home/wmf1997/build_software/extension-template-main/duckdb/third_party/utf8proc/include -I/home/wmf1997/build_software/extension-template-main/duckdb/third_party/concurrentqueue -I/home/wmf1997/build_software/extension-template-main/duckdb/third_party/pcg -I/home/wmf1997/build_software/extension-template-main/duckdb/third_party/tdigest -I/home/wmf1997/build_software/extension-template-main/duckdb/third_party/mbedtls/include -I/home/wmf1997/build_software/extension-template-main/duckdb/third_party/jaro_winkler -I/home/wmf1997/build_software/extension-template-main/duckdb/third_party/yyjson/include -I/home/wmf1997/build_software/extension-template-main/src/include -I/home/wmf1997/build_software/extension-template-main/duckdb/third_party/libpg_query/include -O3 -DNDEBUG -ffunction-sections -fdata-sections -O3 -DNDEBUG   -fPIC   -D _GLIBCXX_USE_CXX11_ABI=0 -std=c++11 -o CMakeFiles/quack_extension.dir/src/quack_extension.cpp.o -c /home/wmf1997/build_software/extension-template-main/src/quack_extension.cpp
/usr/bin/ar qc libquack_extension.a CMakeFiles/quack_extension.dir/src/quack_extension.cpp.o
/usr/bin/ranlib libquack_extension.a

build loadable_extension

/usr/bin/c++ -DDUCKDB_BUILD_LIBRARY -DDUCKDB_BUILD_LOADABLE_EXTENSION -I/home/wmf1997/build_software/extension-template-main/duckdb/src/include -I/home/wmf1997/build_software/extension-template-main/duckdb/third_party/fsst -I/home/wmf1997/build_software/extension-template-main/duckdb/third_party/fmt/include -I/home/wmf1997/build_software/extension-template-main/duckdb/third_party/hyperloglog -I/home/wmf1997/build_software/extension-template-main/duckdb/third_party/fastpforlib -I/home/wmf1997/build_software/extension-template-main/duckdb/third_party/skiplist -I/home/wmf1997/build_software/extension-template-main/duckdb/third_party/fast_float -I/home/wmf1997/build_software/extension-template-main/duckdb/third_party/re2 -I/home/wmf1997/build_software/extension-template-main/duckdb/third_party/miniz -I/home/wmf1997/build_software/extension-template-main/duckdb/third_party/utf8proc/include -I/home/wmf1997/build_software/extension-template-main/duckdb/third_party/concurrentqueue -I/home/wmf1997/build_software/extension-template-main/duckdb/third_party/pcg -I/home/wmf1997/build_software/extension-template-main/duckdb/third_party/tdigest -I/home/wmf1997/build_software/extension-template-main/duckdb/third_party/mbedtls/include -I/home/wmf1997/build_software/extension-template-main/duckdb/third_party/jaro_winkler -I/home/wmf1997/build_software/extension-template-main/duckdb/third_party/yyjson/include -I/home/wmf1997/build_software/extension-template-main/src/include -I/home/wmf1997/build_software/extension-template-main/duckdb/third_party/libpg_query/include -O3 -DNDEBUG -ffunction-sections -fdata-sections -O3 -DNDEBUG   -fPIC -fvisibility=hidden   -D _GLIBCXX_USE_CXX11_ABI=0 -std=c++11 -o CMakeFiles/quack_loadable_extension.dir/src/quack_extension.cpp.o -c /home/wmf1997/build_software/extension-template-main/src/quack_extension.cpp

generate the extension (?without metadata?)

I use absolute path to direct the .a files. The reason is that I build the entire duckdb library in another directory (by using make install, and without any -O3 optimization) at first, and then, at the base of building up my own duckdb, I manage to build the quack extension. (P.S. I deleted some .a files, for example, .a files associated to parquet extension, since I thought they were not related to the quack extension)

/usr/bin/c++ -fPIC -O3 -DNDEBUG -ffunction-sections -fdata-sections -O3 -DNDEBUG  -shared -Wl,-soname,quack.duckdb_extension -o quack.duckdb_extension quack_extension.cpp.o.load /home/wmf1997/build_software/duckdb_build/lib/libduckdb_static.a  -Wl,--gc-sections -Wl,--exclude-libs,ALL /usr/lib/x86_64-linux-gnu/libssl.so /usr/lib/x86_64-linux-gnu/libcrypto.so libquack_extension.a /home/wmf1997/build_software/duckdb_build/lib/libduckdb_static.a libquack_extension.a -ldl /home/wmf1997/build_software/duckdb_build/lib/libduckdb_fsst.a /home/wmf1997/build_software/duckdb_build/lib/libduckdb_fmt.a /home/wmf1997/build_software/duckdb_build/lib/libduckdb_pg_query.a /home/wmf1997/build_software/duckdb_build/lib/libduckdb_re2.a /home/wmf1997/build_software/duckdb_build/lib/libduckdb_miniz.a /home/wmf1997/build_software/duckdb_build/lib/libduckdb_utf8proc.a /home/wmf1997/build_software/duckdb_build/lib/libduckdb_hyperloglog.a /home/wmf1997/build_software/duckdb_build/lib/libduckdb_fastpforlib.a /home/wmf1997/build_software/duckdb_build/lib/libduckdb_skiplistlib.a /home/wmf1997/build_software/duckdb_build/lib/libduckdb_mbedtls.a /home/wmf1997/build_software/duckdb_build/lib/libduckdb_yyjson.a /usr/lib/x86_64-linux-gnu/libssl.so /usr/lib/x86_64-linux-gnu/libcrypto.so 

WMF1997 avatar Oct 06 '24 15:10 WMF1997