"Out-of-tree-extensions" missing for Windows
Hi, Thanks for all the great work on maintaining this package. I can interpret from the work being done on 0.10.0 that this requires some effort, but it's greatly appreciated.
"Out-of-tree-extensions" appear to be missing on Windows as reported in other issues (e.g. #23). I made a quick script to check which extensions are not available based on duckdb_extensions() and came up with the following list for version 0.10.0 and 0.9.2. As pointed out by @Mause, most of these are "Out-of-tree-extensions".
0.10.0
| Extension | Linux | OSX | Windows | ||||
|---|---|---|---|---|---|---|---|
| amd64 | amd64_gcc4 | arm64 | amd64 | arm64 | amd64 | amd64_rtools | |
| arrow | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ |
| autocomplete | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| aws | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ |
| azure | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ |
| excel | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| fts | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| httpfs | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| iceberg | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ |
| icu | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| inet | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| json | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| motherduck | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ |
| mysql_scanner | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ |
| parquet | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| postgres_scanner | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ |
| spatial | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ |
| sqlite_scanner | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| substrait | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ |
| tpcds | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| tpch | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
0.9.2
| Extension | Linux | OSX | Windows | ||||
|---|---|---|---|---|---|---|---|
| amd64 | amd64_gcc4 | arm64 | amd64 | arm64 | amd64 | amd64_rtools | |
| arrow | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ |
| autocomplete | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| aws | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ |
| azure | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ |
| excel | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| fts | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| httpfs | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| iceberg | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ |
| icu | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| inet | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| json | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| motherduck | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ |
| mysql_scanner | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ |
| parquet | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| postgres_scanner | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ |
| spatial | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ |
| sqlite_scanner | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ |
| substrait | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ |
| tpcds | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| tpch | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Most of the extensions you've noted as "missing" are more accurately described as "out-of-tree extensions". The exceptions are the visualizer extension, which has been removed, and the jemalloc extension, which is only ever compiled in, you can't download it
Most of the extensions you've noted as "missing" are more accurately described as "out-of-tree extensions". The exceptions are the visualizer extension, which has been removed, and the jemalloc extension, which is only ever compiled in, you can't download it
Thank you. I figured that was a probable cause for the jemalloc and visualizer extensions. I was not familiar with the definition of "out-of-tree-extensions", but assumed that listed official extensions (e.g. azure, spatial) were intended to be supported by R similarly to Python. Reading up on duckdb/extension, I understand that the build process is more complex than that.
As a note, @Maxxen added spatial build for R in the duckdb/duckdb_spatial repository, it should be available from v0.10.0 onward.
What's the best way forward here? Who could help tick all the boxes?
Is there a workaround to build/install extensions (e.g., azure) on Windows? Does the fix require opening an issue under the relevant extension?
I am also interested in the Azure extension. I do not understand what it takes to enable one of these extensions. Is there a PR that I can look at to see an example? (I am guessing it is way out of my wheelhouse but maybe it will help someone else get started?).
Any updates on this front? Any advice on how to get the Azure extension running in windows?